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THE  USE  OF  SIMULATORS  FOR  TRAINING 
IN-FLIGHT  AND  EMERGENCY  PROCEDURES 


by 


Edward  E.  Eddowes  and  Wayne  L.  Waag 
Operational  Training  Division1 
Air  Force  Human  Resources  Laboratory 
Williams  Air  Force  Base,  Arizona  85224 

USA 


SUMMARY 

The  use  of  simulators  for  training  in-flight  and  emergency  procedures  has  received  Increased  emphasis 
as  a  result  of  the  decreased  availability  and  high  cost  of  fuel.  This  report  examines  the  nature  of 
pilot  skills  and  suggests  a  strategy  for  using  flight  simulators  to  acquire  and  maintain  them.  Salient 
characteristics  of  state  of  the  art  simulators  and  the  visual  display  system  capabilities  they  provide 
for  training  contact  flying  tasks  are  described.  Research  on  transfer  of  learning  from  simulator  to 
aircraft  in  a  variety  of  training  tasks  is  reviewed.  The  use  of  simulators  as  aircraft  substitutes  and 
their  integration  within  an  array  of  ground  training  media  are  compared  and  contrasted  to  illustrate  the 
cost-effectiveness  potential  of  these  two  approaches.  The  rationale  for  and  the  characteristics  of  a 
simulator-oriented  emergency  procedures  training  program  which  emphasizes  pilot  decision  making  skills 
are  presented.  The  development  of  aircrew  performance  measurement  systems  for  use  in  evaluating, 
refining  and  documenting  simulator  training  effectiveness  is  reported.  An  interpretation  of  the  impact 
of  current  developments  on  the  future  use  of  simulators  is  offered  and  a  list  of  conclusions  provided. 


I.  INTRODUCTION  AND  OVERVIEW 


THE  FUEL  RESOURCE  PROBLEM 

The  oil  embargo  following  the  Yom  Kippur  War  in  1973  and  subsequent  establishment  of  a  cartel  by  the 
oil  producing  and  exporting  countries  of  the  Middle  East  focused  Interest  on  the  Increased  costs  and 
decreased  availability  of  fuel  supplies  for  use  in  support  of  military  flying  training.  These 
international  developments  forced  recognition  of  the  fact  that  military  air  forces  would  have  to  find 
alternatives  to  in-flight  training  which  would  Insure  maintenance  of  the  high  levels  of  pilot  skill  and 
mission  readiness  required  by  national  defense  policy.  The  resulting  problem  is  how  to  furnish 
acceptable  training  for  combat  aircrews  to  guarantee  the  required  levels  of  mission  readiness  while 
minimizing  the  use  of  fuel  resources. 

Additional  emphasis  of  the  flying  training  problem  involves  recent  development  of  a  number  of  high 
performance  aircraft  systems,  such  as  the  F-14,  F-15,  F-16,  F-18,  A-10,  and  AWACS  of  the  United  States 
military  air  forces,  which  created  a  requirement  for  additional  transition  and  qualification  training  of 
aircrews  for  these  systems.  The  training  problem  is  further  exacerbated  by  the  relatively  low  flying 
experience  levels  of  today's  military  pilots  compared  with  the  flying  experience  of  the  pilots  of  ten 
and  twenty  years  ago. 

SIMULATION  AS  A  SOLUTION 

Approaches  to  solving  the  array  of  flying  training  problems  have  appeared  with  the  development  of 
state  of  the  art  high  capability  simulators,  equipped  with  visual  display  systems  which  furnish 
opportunities  for  simulator  training  in  a  variety  of  contact  flying  tasks  for  the  first  time.  These  new 
flight  simulators  have  demonstrated  a  potential  to  generate  substantial  transfer  of  training  to  the 
aircraft  (1).  Meanwhile,  research  in  training  methods  and  media  has  shown  how  the  new  flight  simulators 
may  be  used  to  Increase  their  training  effectiveness,  extend  their  utility  over  a  broader  range  of 
flying  training  requirements  and,  as  a  consequence,  further  improve  their  potential  to  supplement  flying 
training  in  the  aircraft  (2).  The  high  cost  of  new  aircraft  systems  and  the  Increasing  cost  of 
operating  them  thus  may  be  partially  counterbalanced  through  use  of  the  efficient  and  economical 
training  methods  and  devices  currently  being  prepared  for  integration  within  tomorrow's  military  flying 
training  programs. 

Until  recently,  training  managers  have  regarded  flight  simulators  as  aircraft  substitutes. 
Simulators  have  been  designed  to  be  as  much  like  the  aircraft  as  possible.  The  trainee  then  could  fly 
the  simulator  as  he  would  fly  the  aircraft  and  the  simulator's  training  value  would  be  judged  on  the 
basis  of  how  much  like  the  aircraft  it  was  and  its  effectiveness  would  be  calculated  in  terms  of  the 
flying  hour  reduction  in  a  training  program  in  which  it  was  used. 

NEM  ROLE  OF  THE  SIMULATOR 

A  shift  in  emphasis  of  the  role  of  the  simulator  in  flying  training  has  recently  emerged  (3). 
According  to  this  view,  the  flight  simulator  1$  seen  as  supporting  the  acquisition/maintenance  of  flying 
skill  rather  than  providing  a  substitute  in-flight  environment  on  the  ground.  Thus,  the  simulator  may 
be  used  to  Improve  a  trainee's  readiness  to  take  maximum  advantage  of  subsequent  training/practice 
opportunities  and  becomes  one  of  an  array  of  training  devices  which  vary  in  their  cost,  complexity, 
capabilities  and  representativeness  from  those  of  the  academic  classroom  through  audio-visual  materials, 
video  tape,  film,  part-task  training  and  mission  simulators  to  me  aircraft  (4).  In  addition,  the  new 
emphasis  of  flying  training  simulation  defines  the  value  of  any  training  device  in  terms  of  Its  training 
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effectiveness,  that  is.  Its  effectiveness  In  providing  practice  opportunities  which  lead  to  the 
acquisition  of  transferable  skills  (1,  3). 

OVERVIEW 

Before  proceeding  further  with  this  report  on  the  use  of  simulators  for  training  In-flight  and  emer¬ 
gency  procedures,  It  may  be  useful  to  look  ahead  to  the  series  of  upcoming  topics  to  aid  the  reader  to 
see  the  Interrelationships  among  and  the  cumulative  meaning  of  the  various  sets  of  facts  and 
Interpretations  to  be  presented.  Given  the  flying  training  problem  created  by  the  growing  fuel  shortage, 
the  Increased  complexity  and  sophistication  of  today's  high  performance  aircraft  systems,  coupled  with 
the  decreased  flying  experience  levels  of  current  aircrew  personnel,  the  following  chapters  will  seek  to 
formulate  a  solution  by  generating  compelling  new  combinations  of  older  (and  some  newer)  Ideas. 

In  Chapter  2,  the  essence  of  the  pilot's  flying  skill  Is  described  to  serve  as  the  basis  for 
understanding  what  a  pilot  must  learn  during  his  flying  training.  The  analysis  explores  the  need  for  a 
pilot  to  look  ahead  to  develop  his  aircraft  control  requirements  as  an  Inevitable  corollary  of  operating 
an  aircraft  In  an  elastic  medlun.  The  result  of  this  Interpretation  Is  a  picture  of  the  pilot  as  flight 
planner,  anticipator  and  detector  of  problems  and  finely-tuned  Information  processor  dedicated  to  safe, 
successful  achievement  of  mission  objectives. 

With  a  cognitively  oriented.  Information  processing  concept  of  the  pilot's  task  In  mind,  a  strategy 
for  using  simulators  In  flying  training  Is  developed  In  Chapter  3  In  which  ground  training  Is  seen  as 
preparing  and  familiarizing  the  learner  with  his  task  so  he  can  make  the  most  of  his  training 
opportunities  In  the  aircraft.  Specifically,  the  concept  of  the  simulator  as  aircraft  substitute  Is 
shown  to  foster  development  of  more  costly  and  complex  training  devices  than  are  required  to  achieve 
effective  transfer  of  learning  from  simulator  to  aircraft.  Conversely,  use  of  flight  simulators  to 
prepare  the  student  to  take  greater  advantage  of  his  subsequent  flight  training  leads  to  more  effective 
use  of  simulator  capabilities  and  permits  development  of  simpler,  less  expensive  devices  which  no  longer 
must  replicate  the  aircraft  in  every  way  possible  to  achieve  training  effectiveness. 

Chapter  4  traces  the  growth  and  development  of  simulation  In  military  aviation  training  from  the 
early  days  of  behind- the- lines  training  In  obsolete  aircraft  to  the  structured  flying  training  programs 
of  today.  Increased  Implementation  of  ground-based  training  programs  has  been  paralleled  and  preceded 
by  continuous  advances  In  the  state  of  the  art  of  flight  simulation  equipment.  Salient  features  of 
current  flight  simulators.  Including  the  use  of  digital  computers  to  control  operation  of  the  aircraft 
cockpit  equipment,  flight  dynamics  math  model  and  visual  and  motion/force  cueing  systems  are  described 
briefly  to  Illustrate  how  more  and  more  contact  flying  tasks  are  being  trained  In  simulators  as  their 
capabilities  to  support  such  additional  training  are  demonstrated. 

Following  the  brief  description  of  state  of  the  art  simulators,  the  results  of  major  studies  of 
transfer  of  learning  from  simulator  to  aircraft  are  reviewed  In  Chapter  5.  In  this  review,  which 
Includes  an  explication  of  the  limitations  of  existing  data  and  the  several  problems  associated  with 
transfer  studies  In  flying  training,  a  picture  of  the  effectiveness  of  simulator  training  emerges  In 
which  there  are  no  Instances  of  significant  negative  transfer.  Such  an  outcome  Is  Interpreted  as 
suggesting  that  simulator  training  In  which  flying  tasks  can  be  practiced  and  learned  will  lead  to 
Improved  trainee  performance  In  the  aircraft. 

A  description  of  the  distinctive  characteristics  of  the  employment  of  flight  simulators  In  two 
current  training  programs  Is  provided  In  Chapter  6.  The  roles  of  the  simulator  In  the  US  Air  Force's 
Air  Training  Command  Undergraduate  Pilot  Training  program  and  the  American  Airlines'  Pilot  and  Copilot 
Qualification  Training  program  are  compared  and  contrasted.  Examination  of  these  different  programs 
Illustrate  how  use  of  the  simulator  as  aircraft  substitute  and  the  Integration  of  the  simulator  within 
an  array  of  ground  training  media  have  demonstrated  their  cost  effectiveness. 

While  the  employment  of  simulators  In  emergency  procedures  training  has  been  one  of  the  constants  In 
flying  training  programs  virtually  from  the  beginning.  In  Chapter  7  a  new  and  slightly  different  approach 
is  described.  The  typical  emergency  procedure  Is  practiced,  learned  and  then  tested  In  a  simulator 
because  It  cannot  be  practiced  safely  In  the  aircraft.  In  the  new  approach,  referred  to  here  as  Situa¬ 
tional  Emergency  Training,  the  emphasis  has  been  shifted  from  the  fast,  accurate  execution  of  a  perfectly 
memorized  sequence  of  actions  to  a  judgmental,  problem  solving  process  by  the  aircrew  member  confronted 
with  an  emergency  situation.  Situational  Emergency  Training  focuses  on  the  pilot’s  analyzing  the  situa¬ 
tional  aspects  of  the  emergency  condition  In  the  context  of  his  mission,  deciding  what  to  do  to  deal  with 
the  problem  and  taking  the  appropriate  actlon(s)  so  that  he  may  make  a  safe  landing  as  soon  as  possible. 

The  final  substantive  chapter  of  this  report  describes  progress  made  In  the  development  of  automated 
pilot  performance  measurement  systems  for  Implementation  In  current  flight  simulators.  The  goal  of 
these  simulator  performance  measurement  system  development  efforts  Is  to  provide  objective,  quantitative 
evidence  of  the  result  of  simulator  training  for  use  by  students.  Instructors,  training  managers, 
researchers,  simulator  designers  and  procurement  officers.  The  objective,  quantitative  data  generated 
by  the  simulator  performance  measurmaent  systems  then  nay  be  applied  to  evaluate  and  document  the 
success  of  students,  guide  instructors,  help  training  managers  fine-tune  their  programs,  and  aid 
researchers,  designers  and  procurement  officers  make  efficient  decisions  about  what  Is  required  and 
which  alternative  will  provide  the  best  training  at  the  lowest  cost. 

In  the  last  chapter  a  projection  of  the  future  use  of  simulators  In  flying  training  programs  and  a 
list  of  conclusions  based  on  and  supported  by  the  facts  and  Ideas  presented  In  the  preceding  chapters 
are  offered  In  an  Interpretation  or  the  impact  of  current  developments  on  the  future  use  of  flight 
simulators.  Here  a  picture  of  future  flying  training  Is  developed  In  which  more  different  kinds  of 
simulators  ere  used  to  satisfy  a  wider  variety  of  training  requirements  with  Increased  efficiency  and 
economy.  Improved  organization  and  Integration  of  tomorrow's  simulator  and  aircraft  training  also  are 
projected  as  a  consequence  of  the  greater  realization  of  flight  simulators'  training  potential  through 


application  of  modern  training  methods  and  the  refinement  of  current  programs  made  possible  by 
exploiting  the  Implementation  of  automated  aircrew  performance  measurement  systems. 

A  list  of  conclusions  Is  presented  to  close  this  report.  These  conclusions  reflect  both  the  extent 
and  the  limitation  of  Information  describing  the  current  use  of  simulators  for  In-flight  and  emergency 
procedures  training. 


II.  THE  ESSENCE  Of  THE  PILOT'S  FLYING  SKILL 


CHARACTERISTICS  OF  FLIGHT  OPERATIONS 

Basic  characteristics  of  aircraft  operations  will  be  examined  In  this  chapter  to  develop  a  descrip¬ 
tion  of  the  essence  of  a  pilot's  flying  skill  which  transcends  the  explanatory  limits  of  an  older  notion 
of  pilot  skill  as  hand-eye  coordination.  Before  proceeding  with  this  Investigation,  the  Implications  of 
controlling  an  aircraft's  flight  path  in  three-dimensional  space  will  be  considered,  and  the  implications 
of  the  requirement  for  developing  plans  to  accommodate  a  variety  of  conditions  which  may  arise  during  a 
flight  will  be  described  to  establish  a  starting  point  for  the  analysis  which  follows. 

Aircraft  operate  In  an  elastic  medium  and  are  controlled  by  means  of  a  pilot's  manipulation  of  the 
aircraft's  control  surfaces.  The  speeds  at  which  all  aircraft  normally  operate  prevent  abrupt  changes 
in  flight  path  and  require  that  such  changes  be  accomplished  gradually  and  smoothly  enough  to  avoid 
departure  from  controlled  flight.  Thus,  the  pilot  must  know  or  become  familiar  with  the  maneuvering 
characteristics  of  an  aircraft  and  be  able  to  control  Its  flight  path  as  required  to  accomplish  all 
maneuvers  within  Its  performance  capabilities  to  achieve  the  objectives  of  a  flying  maneuver  or 
m1ss1on2. 

In  addition  to  mastering  flight  maneuvers,  a  pilot  must  know  the  requirements  for  controlling  an 
aircraft  throughout  a  mission  or  he  must  have  such  requirements  information  available  to  guide  him  In 
controlling  the  aircraft's  flight  path  and  managing  its  operation.  Because  flying  operations  often  are 
conducted  under  variable  environmental  conditions,  it  is  a  practical  necessity  that  a  pilot  know  of  a 
number  of  alternative  means  which  can  be  used  to  achieve  mission  objectives  if  for  any  reason  prior 
plans  or  requirements  Information  no  longer  apply  during  a  mission  due  to  environmental  or  aircraft 
status  changes.  Consequently,  pilots  are  trained,  qualified  and  periodically  evaluated  on  the 
discipline,  accuracy,  efficiency,  completeness  and  safety  of  the  aircraft  control  and  system  management 
procedures  they  use  in  accomplishing  flying  missions. 

THE  PILOT'S  DYNAMIC  ENVIRONMENT 

As  a  result  of  operating  an  aircraft  In  a  dynamic,  three-dimensional  environment  and  practicing 
flying  maneuvers,  a  pilot  learns  and  knbws  the  maneuvering  characteristics  of  his  aircraft  and,  given 
sufficient  Information  and  understanding  of  the  requirements  of  a  mission,  can  plan  (formally  or 
informally)  what  he  must  do  to  satisfy  the  mission  requirements  (5).  Through  his  familiarity  with  his 
aircraft  and  knowledge  of  the  mission  requirements,  a  pilot  can  satisfy  the  requirements  accurately, 
efficiently  and  safely.  To  deal  effectively  with  his  momentary  aircraft  control  tasks,  the  pilot  must 
project  the  current  status  of  his  aircraft  into  the  near  future  time-space,  evaluate  this  projection  in 
terms  of  the  remaining  mission  requirements  and  take  appropriate  control  actions  to  continue  satisfying 
them.  This  kind  of  behavioral  process  has  been  described  by  Hull  (6)  and  may  be  assumed  to  support  the 
pilot's  projection  of  current  aircraft  states  Into  the  future  In  order  to  determine  his  aircraft  control 
action  requirements. 

If/when  an  error  Is  projected,  the  pilot  will  make  the  control  Inputs  he  has  decided  will  reduce  to 
a  practical  mlnimuii  the  error  remaining  after  he  reprojects  the  consequences  of  his  corrective  action 
ahead  into  the  next  mission  Increment.  The  detection  of  an  existing  error  or  an  error  emerging  from  the 
projected  future  aircraft  status  serves  to  trigger  this  behavioral  process  rather  than  merely  Informing 
the  pilot  of  the  error  or  the  amount  of  error  to  be  eliminated  and  Illustrates  the  pilot’s  predominantly 
looking-ahead  orientation  (7).  Thus  the  pilot  uses  the  current  state  of  his  aircraft  not  as  the  basis 
for  correcting  errors  If  they  are  present,  but  Instead  projects  the  current  state  ahead  and  makes  his 
subsequent  control  actions  based  on  the  projection  and  what  must  be  done  to  optimize  the  future  state  of 
the  aircraft  with  respect  to  remaining  mission  requirements  (8,  9). 

THE  PILOT'S  MISSION  PLAN 

Having  established  that  the  pilot  bases  his  aircraft  control  actions  on  projections  of  his  current 
state.  It  can  be  seen  that  ordinarily  he  is  not  responding  to  any  existing  stimulus  pattern  when  a 
control  action  response  Is  made.  Instead  the  pilot  responds  to  his  projection  of  the  future  state  of 
the  aircraft  or  to  an  Idea  referred  to  here  as  his  mission  cognitive  structure  or  plan  which  represents 
the  mission  and  his  aircraft's  position  In  this  mission. 

Given  the  assumption  of  a  pilot  with  a  mission  plan  to  guide  him  In  his  manipulation  of  the  aircraft 
during  a  flight.  It  Is  easily  seen  that  the  skilled  pilot  will  bring  to  his  mission  planning  those  of 
his  previous  experiences  he  believes  to  be  relevant.  As  he  begins  to  develop  his  plan,  a  pilot  will 
consider  and  evaluate  an  array  of  potential  procedures  and  maneuvers  with  which  he  can  achieve  his 
mission  goals.  From  the  array  of  alternatives,  the  pilot  will  select  those  which  appear  to  represent 


The  term  mission  will  be  used  In  place  of  flying  task  or  flying  maneuver  because  It  covers  a  greater 
range  of  flight  operations  and  because  what  Is  valid  for  flying  tasks  or  maneuvers  Is  assumed  to 
hold  for  missions. 
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the  best  mission  plan.  For  example,  he  may  consider  duration  of  the  flight,  fuel  available,  weather, 
load  data,  takeoff  and  landing  conditions,  terrain  to  be  overflown,  possible  emergency  conditions  and 
alternative  means  for  dealing  with  them.  In  addition,  pilots  typically  consider  other  information  which 
appears  to  be  relevant  based  on  their  perception  of  the  circumstances  of  the  mission  In  order  to  tailor 
the  plan  to  what  they  believe  is  an  optimum  combination  of  their  capabilities  and  mission  requirements. 

AIRMANSHIP 

The  completeness,  elaborateness,  accuracy  and  adequacy  of  the  mission  plan  on  which  a  pilot  relies 
during  a  mission  often  furnishes  an  Index  of  his  experience  and  skill,  sometimes  referred  to  as  his 
airmanship.  These  characteristics  of  a  pilot's  mission  plan  also  furnish  an  Index  of  the  probable 
success  of  his  mission.  A  pilot  whose  missions  plans  are  relatively  complete  and  comprehensive  will  be 
more  likely  to  encounter  fewer  surprises  during  a  mission,  and  because  of  his  greater  preparation  can  be 
expected  to  deal  with  such  unplanned  events  or  emergencies  as  he  may  experience  with  less  difficulty 
than  a  pilot  less  well  prepared. 

With  the  more  experienced  pilot  whose  mission  plans  are  relatively  complete  and  comprehensive.  It 
can  be  assumed  that  he  will  make  appropriate  use  of  the  plan  during  a  mission  to  maximize  the 
smoothness,  accuracy  and  eventual  success  of  the  flight.  Therefore,  at  any  Instant  during  a  mission,  a 
highly  skilled  pilot  may  be  expected  to  be  dealing  with  the  projection  ahead  of  as  many  of  the 
alternatives  considered  in  developing  the  mission  plan  as  he  believes  relevant  or  applicable  (10). 

SITUATION  AWARENESS 

If  an  increment  of  the  pilot's  mission  plan  is  taken  as  a  point  of  reference,  the  experienced  pilot 
can  be  expected  to  exhibit  what  has  come  to  be  known  as  situation  awareness,  a  comprehensive 
appreciation  of  what  is  going  on  and  what  will  happen  next.  That  is,  because  of  the  completeness  of  his 
mission  plan  and  his  projection  ahead  of  the  relevant  alternatives,  his  situation  awareness  will  be 
relatively  high  at  any  instant  (11).  In  addition,  we  can  see  that  the  quality  of  the  pilot's  situation 
awareness  is  related  directly  to  the  quality  of  the  plan  he  has  developed,  his  understanding  of  mission 
requirements  and  his  ability  to  satisfy  them. 

If  this  rationale  is  valid,  it  can  be  concluded  the  pilot's  task  is  essentially  cognitive  in 
nature.  It  can  be  concluded  further  that  the  development  of  the  pilot's  skill  may  profitably  be 
organized  around  his  mission  planning  task  requirements  in  order  to  facilitate  the  growth  of  his 
situation  awareness,  probably  the  best  single  indicator  of  his  general  overall  pilot  ability,  or 
airmanship. 

SUBSYSTEMS  MANAGEMENT  AND  AIRCRAFT  CONTROL 

The  analysis  of  the  essence  of  the  pilot’s  flying  skill  presented  here  focuses  on  aircraft  control 
tasks  because  the  requirement  for  pilot  cognitive  behavior  is  more  obvious  with  them  than  with  the  pilot 
skill  involved  in  operation  of  such  on-board  equipment  as  the  communication,  navigation  or  other 
electronic  subsystems.  A  close  look  at  management  of  aircraft  subsystems,  however,  will  reveal  that  the 
pilot's  performance  requirements  for  their  operation  are  very  much  like  those  involved  in  aircraft 
control  except  that  they  are  characteristically  more  step-by-step  and  less  continuous  in  nature.  The 
similarity  may  be  clearly  seen  by  considering  the  timing,  sequencing  and  integrating  aspects  of  the 
subsystems  management  task. 

Thus  it  appears  that  the  key  to  a  pilot's  effectiveness  lies  in  his  ability  to  develop  comprehensive 
mission  plans  and  to  use  them  in  order  to  know  what  to  look  for,  where  and  when  to  look  for  it,  and  what 
to  do  with  it  once  he  finds  it  in  controlling  and  managing  his  aircraft  system  during  a  mission  (12). 
It  appears  also  that  these  essential  pilot  skills  involve  the  seeking  and  processing  of  information  and 
represent  primarily  the  operation  of  his  repertory  of  cognitive  processes  (13). 


III.  A  STRATEGY  FOR  USING  SIMULATORS  IN  FLYING  TRAINING  PROGRAMS 


SIMULATOR  FIOELITY 

It  is  axiomatic  that  performance  on  a  task  is  facilitated  if  the  acquisition  of  ability  to  perform 
the  task  is  accomplished  in  a  learning  environment  similar  or  identical  to  the  environment  in  which 
performance  on  the  task  will  be  evaluated.  In  flying  training,  this  compelling  notion  has  stimulated  an 
unceasing  search  for  a  learning  environment  for  student  pilots  as  much  like  the  performance  environment, 
the  aircraft,  as  state  of  the  art  flight  simulator  technology  can  provide. 

This  state  of  affairs  has  led  to  development  of  simulators  which  incorporate  as  many  aircraft-like 
features  as  possible.  Simulators  which  are  more  like  the  aircraft,  as  determined  by  quantitative 
measures/the  ratings  of  experienced  alrcrewmen  are  said  to  be  more  realistic  or  have  higher  fidelity. 
Conversely,  simulators  which  are  less  like  the  aircraft  are  identified  as  less  realistic  or  lower 
fidelity  devices.  Accordingly,  it  is  assumed  that  more  realistic/higher  fidelity  simulators  are  better, 
more  effective  training  devices  and  vice  versa.  Thus  the  search  for  more  realism  and  higher  fidelity 
continues  (3). 

In  developing  a  strategy  for  using  simulators  in  flying  training  which  goes  beyond  the  concept  of 
simulator  reailsm/fidellty  in  which  more  is  better  and  most  is  best,  it  will  be  necessary  to  consider  a 
n«ber  of  ideas  which  bear  on  simulator  utilization.  They  will  then  be  Integrated  into  a  new  whole, 
which,  it  is  hoped  will  possess  greater  explanatory  power  than  the  realism  principle. 
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BASIC  LEARNING  PROCESSES 

It  may  be  profitable  to  note  at  the  beginning  that  learning  is  an  active,  automatic,  continuous  and 
cumulative  process.  Learning  is  not  something  done  to  a  learner.  It  is  the  student  who  learns,  more 
than  it  is  the  instructor  who  teaches.  In  addition,  it  should  be  pointed  out  that  learning  isg't 
guaranteed  through  use  of  a  collection  of  instructional  paraphernalia.  Instructors  may  foster  learning 
more  effectively  by  arranging  and  sequencing  the  presentation  of  appropriate  training  environments  than 
by  use  of  training  aids  and  devices.  It  has  been  observed  that  any  performance  that  can  be  practiced 
can  be  learned  and  any  performance  that  can  be  learned  can  be  transferred  (2).  This  suggests  that  as  a 
learner  acquires  the  ability  to  perform  a  task,  he  becomes  more  able  to  learn  succeeding  training  tasks 
more  easily.  That  is,  mastery  of  later  tasks  is  built  on  the  foundation  of  earlier  learning. 

This  building  block  approach  is  an  application  of  the  concept  of  the  matching  of  training  task 
requirements  or  demands  to  the  individual  trainee's  level  of  ability  so  that  the  next  training  task  is 
slightly  more  difficult  than  the  last  (14).  Thus  the  learner  is  stimulated  to  attempt  the  next  training 
task  without  unacceptable  risk  of  failure.  When  the  problem  of  the  match  of  task  difficulty  and  trainee 
ability  has  been  solved  satisfactorily,  the  sequence  and  content  of  successive  training  tasks  will  build 
the  trainee's  skill  and  his  confidence  in  his  ability  to  exercise  his  skills  in  attacking  successive 
training  problems.  A  diagram  of  this  hypothesized  process  is  shown  in  Figure  1. 


FIG.  1  THE  PROCESS  OF  DEVELOPING  CONFIDENCE 
IN  ONE’S  ABILITY  TO  PERFORM  A  TASK 
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IMPLICATIONS  OF  SIMULATOR  REALISM/FIDELITY 

With  these  basics  In  mind,  the  simulator  realism/fidelity  issue  can  be  examined  further.  It  Is  a 
fact  that  current  flight  simulators  are  more  expensive  than  older  training  devices.  And  it  is  a  fact 
that  they  are  significantly  more  capable  of  supporting  a  wider  variety  of  flying  training  tasks  than 
older  flight  simulators  and  are  generally  regarded  as  more  realistic,  higher  fidelity  devices.  These 
facts  are  taken  as  evidence  that  simulator  capability  and  realism/fidelity  are  positively  and  closely 
related  to  simulator  cost. 

Because  realism  Is  an  Important  factor  In  the  design  of  simulators,  there  Is  a  tendency  to  develop 
devices  which  will  simulate  as  much  of  a  complete  aircraft  mission  as  possible.  This  combines  with  the 
growing  mission  capabilities  of  current  aircraft  and  results  In  more  complex  and  more  costly  simulators. 
Given  the  mission  simulation  capabilities  state  of  the  art  devices  possess.  It  can  be  seen  then  that  In 
any  single  training  session,  typically  focused  on  Individual  training  tasks,  It  Is  unlikely  that  more 
than  a  small  proportion  of  the  simulator's  mission  capabilities  will  be  exercised.  In  addition,  Poven- 
mire  and  Roscoe  (15)  have  shown  that  training  In  a  simulator  is  more  effective  prior  to  the  student's 
first  practice  of  his  training  In  the  aircraft  and  that  Its  effectiveness  diminishes  thereafter. 

It  can  be  seen  also,  assailing  typical  budgetary  constraints,  that  Increasing  slmultor 
realism/fidelity  and  Incurring  the  higher  costs  Involved  will  lead  to  fewer  devices  being  procured, 
fewer  pilots  trained  In  them  and  fewer  opportunities  for  the  capabilities  of  the  simulators  to  be  used 
for  training  purposes.  This  rationale  suggests  the  search  for  more  and  more  realism  and  fidelity  may  be 
self-defeating  as  far  as  overall  cost-effective  simulator  training  Is  concerned. 
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One  final  exaiqple  of  the  paradox  of  simulator  fidelity  warrants  attention  at  this  point.  The 
principal  reason  for  including  a  relatively  costly  cockpit  motion  system  in  a  flight  simulator  is  that 
the  aircraft  moves  in  flight  and  therefore  the  simulator  should  be  capable  of  moving  like  the  aircraft 
in  order  to  be  more  realistic  and  thus  provide  more  effective  training.  It  is  obvious,  however,  that 
all  cockpit  motion  systems  are  Inherently  Incapable  of  providing  realistic  or  high  fidelity  motion  cues 
since  they  are  all  invariably  fastened  to  the  ground  and  cannot  accurately  duplicate  aircraft  motion 
during  flight,  other  than  for  relatively  mild  maneuvers  in  which  pitch  and  roll  rates  are  low  and  of 
brief  duration  (16,  17).  Research  data  on  the  training  effectiveness  of  cockpit  motion  systems  will  be 
reviewed  later  in  this  report. 

DEVELOPING  A  SIMULATOR  STRATEGY 

The  above  considerations,  on  which  the  development  of  a  strategy  for  using  simulators  in  flying 
training  is  based,  can  be  sumnarized  as  follows: 

1.  While  it  is  intuitively  obvious  that  the  more  a  simulator  is  like  an  aircraft,  the  more  likely 
it  will  be  an  effective  training  device;  it  is  also  clear  that  as  the  operational  characteristics  of  a 
simulator  approach  those  of  an  aircraft,  the  conplexity  and  cost  of  the  device  will  increase. 

2.  As  the  capabilities  of  a  simulator  to  support  full  mission  training  increases,  the  difficulties 
of  exercising  all  of  its  capabilities  in  any  training  period  increase  and  its  cost-effectiveness  will 
decrease. 


3.  Rather  than  using  a  realism/f idelity  criterion  for  evaluating  simulator  training  value, 
determine  its  training  effectiveness  by  means  of  a  transfer  of  training  test. 

One  integration  of  these  ideas  is  a  concept  of  least-cost,  sequenced,  multimedia  training, 
diagramed  in  Figure  2.  This  concept  applied  to  flying  training  leads  to  an  array  of  training  aids  that 
increase  in  their  conplexity  and  representativeness  to  provide  a  series  of  training  environments  matched 
to  the  trainee's  discriminative  and  cognitive  readiness  to  deal  with  them  (4).  Figure  2  shows  an  array 
of  training  media  that  begins  with  conventional  academic  instruction  and  ends  each  stage  of  training, 
e.g,,  the  pre-solo,  advanced  contact,  instnnents,  navigation  and  formation  stages  of  undergraduate 
pilot  training  (18),  with  a  criterion  objective  performance  evaluation  in  the  aircraft.  The  least-cost, 
sequenced,  multimedia  model  is  based  on  the  building  block  approach  to  skill  acquisition  and  on  the 
obvious  economy  and  ease  of  scheduling  training  aids  by  using  the  simplest  and  least  expensive  media 
possible  at  all  stages  of  training.  In  addition,  it  accomodates  the  results  of  Fleishman's  (19)  factor 
analytic  studies  of  performance  at  different  stages  of  learning.  A  by-product  benefit  of  this  model  is 
that  it  requires  less  expensive  training  media  and  fewer  high  fidelity,  full  mission  simulators  and 
consequently  may  be  acceptably  cost-effective. 


FIG.  2  LEAST-COST,  SEQUENCED,  MULTI-MEDIA  TRAINING 
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This  approach  Is  one  form  of  a  flying  training  program  which  would  result  from  the  straightforward 
application  of  Instructional  System  Development  (IsD)  procedures  as  described  In  US  Air  Force  Pamphlet 
50-58  (20).  It  will  be  used  as  a  conceptual  framework  for  the  review  of  behavioral  research  presented 
in  subsequent  chapters.  Before  considering  research  evaluations  of  the  training  effectiveness  of  flying 
training  simulators,  however,  it  will  be  useful  to  describe  briefly  some  of  the  salient  characteristics 
of  current  state  of  the  art  devices  In  the  next  chapter. 


IV.  CHARACTERISTICS  OF  STATE  OF  THE  ART  FLIGHT  SIMULATORS 


DEVELOPMENT  OF  FLIGHT  SIMULATION  TRAINING 

Military  training  managers  have  understood  intuitively  that  combat  is  the  only  reality  and 
historically  they  have  simulated  the  ttle  environment  by  using  combat  equipment  for  training 
purposes.  This  represents  an  initial  development  of  a  training  simulation  and  the  concept  subsequently 
has  been  accepted  as  necessary  and  sufficient  for  effective  aircrew  training. 

During  World  War  I,  flying  training  took  place  at  staging  airfields  behind  the  front  lines  using 
operational  as  well  as  obsolete  aircraft.  Special  flying  schools  were  set  up  further  from  the  combat 
zone  where  basic  training  was  given  in  a  variety  of  obsolete  aircraft  (21,  22,  23).  The  same  mode  of 
training  was  carried  forward  during  World  War  II  (24).  In  addition,  the  use  of  segments  of  operational 
equipment  began  to  appear  in  aircrew  training  programs.  Late  in  World  War  II,  rudimentary  flight 
simulators  were  developed  and  the  expansion  of  ground  training  as  a  component  in  flying  training 
programs  began  (25). 

Since  the  initial  appearance  of  the  flight  simulator,  there  has  been  a  steady  proliferation  of 
instructional  media  applied  in  flying  training  programs.  Success  of  many  of  these  applications  has  been 
documented,  for  example,  by  Reid  and  Cyrus  (26),  Eddowes  et  al.  (4),  and  Smith,  Waters  and  Edwards  (27). 

CURRENT  FLIGHT  SIMULATOR  CAPABILITIES 

It  is  a  challenge  for  current  aircrew  training  technology  to  provide  the  elements  of  the  training 
environment  that  will  most  effectively  and  economically  facilitate  acquisition  and  maintenance  of  the 
performance  capabilities  needed  to  meet  combat  mission  requirements.  Today's  flight  simulators  possess 
significantly  greater  capabilities  for  supporting  aircrew  training  than  those  of  previous  generation 
devices.  The  new  simulators,  when  equipped  with  visual  display  systems,  can  generate  training 
environments  in  which  aircrews  can  acquire  and  practice  normal,  emergency  and  combat  maneuvers.  These 
devices  also  can  be  used  to  support  training  in  basic  and  advanced  contact  flying  and  instrument  and 
navigation  procedures. 

Through  incorporation  of  a  variety  of  advanced  instructional  features,  such  as  problem  freeze,  per¬ 
formance  replay,  malfunction  insertion  and  automated  performance  measurement,  current  simulators  provide 
capabilities  needed  for  improved  instruction.  In  addition  to  furnishing  training  and  practice  in  flying 
maneuvers  which  cannot  or  need  not  be  taught  in  the  aircraft,  state  of  the  art  simulators  can  be  used  as 
teaching  tools  rather  than  as  substitute  aircraft.  They  can  prepare  aircrew  trainees  to  take  maximum 
advantage  of  their  next  training  tasks,  whether  they  are  presented  in  another  simulator  or  an  aircraft. 

The  increased  training  capabilities  of  modern  flight  simulators  are  the  result  of  synergistic 
integration  of  digital  computer,  visual  display  and  motion  cueing  systems.  Simulator  operations 
controlled  by  digital  computers  have  gradually  replaced  older  analog  and  hybrid  systems.  They  have 
achieved  greater  speed,  accuracy  and  range  of  flight  maneuvers  that  can  be  simulated;  and,  in  addition, 
permit  easier,  less  expensive  and  faster  modification  when  it  is  necessary,  for  example,  to  update  the 
aircraft  flight  performance  characteristics  or  to  change  the  operation  of  the  whole  system  to  improve 
its  training  potential. 

The  inclusion  of  motion  cueing  systems  in  state  of  the  art  flight  trainers  typically  involves  one  of 
a  variety  of  motion  simulation  mechanisms  which  generate  onset  cues  as  a  maneuver  is  initiated,  and, 
subsequently  ”wash-out“3  the  onset  cues  as  a  stabilized  state  is  attained.  There  are  a  number  of 
types  of  cockpit  motion  mechanization  schemes  and  a  variety  of  means  for  actuating  them.  An  alternative 
motion  cueing  system,  the  g-seat,  has  recently  been  developed  and  used  in  several  simulator  requirements 
studies  (28,  29).  In  g-seat  systems,  air  or  fluid  is  used  to  control  the  inflation  and  deflation  of 
cells  in  the  aircrew  seat  pan  and  seat' back  panels  to  create  and  relieve  pressure  on  the  pilot's  back, 
buttocks  and  thighs,  analogous  to  sustained  g-forces  during  higher  performance  maneuvers  characteristic 
of  air- to- surface  weapon  delivery  and  air-to-air  combat  maneuvers. 

The  development  and  incorporation  of  wide  fleld-of-vlew  (FOV)  visual  displays  within  modern  flight 
simulators  is  a  significant  factor  in  the  dramatic  increase  in  simulator  training  capabilities.  These 
visual  displays  typically  rely  on  the  control  capabilities  provided  by  the  digital  computers  with  which 
virtually  all  current  simulators  are  equipped.  There  are  three  major  types  of  visual  display  schemes  for 
simulators:  television-model  board,  projected  imagery,  and  computer-generated  imagery.  Although  hybrid 
combinations  of  the  three  types  have  been  and  are  being  developed,  the  following  description  of  simulator 
visual  display  systems  will  focus  on  their  salient  characteristics. 


3  "Wash-out"  refers  to  the  gradual  removal  of  a  roll/pitch  onset  motion  cue  at  such  a  low  rate  that  a 
pilot  will  not  notice  the  "wash-out"  motion  of  the  simuator  cockpit. 


TV-model  board  systems  are  the  oldest  of  current  simulator  visual  display  systems.  They  involve  a 
model  terrain  board  and  a  TV  camera  which  is  driven  across  the  model  board  by  the  simulator's  computer 
as  the  aircraft  it  simulates  performs  various  contact  maneuvers.  The  imagery  picked  up  by  the  camera  is 
projected  onto  a  viewing  screen  in  front  of  the  pilot/copilot  stations,  TV-model  systems  have  been 
widely  used  to  provide  a  visual  display  of  the  landing  maneuver  for  instrument  letdown  and  approach 
training.  Such  systems  focus  on  the  relatively  low  performance  maneuvering  associated  with 
around-the-airport  contact  training.  The  training  capabilities  of  TV-model  visual  systems  are  limited 
by  the  size  and  content  of  the  model  board  which  in  turn  determines  the  contact  maneuvers  which  can  be 
practiced  within  the  simulated  visual  environment. 

Projected  imagery  displays  involve  the  use  of  either  a  film,  a  model  or  a  corrputer-generated  image 
to  furnish  the  basic  display  input,  which  is  processed  electro-optical ly  and  projected  onto  a  curved  or 
dome-shaped  screen.  The  distance  from  the  aircrew  station  is  such  that  the  pilot  views  the  display  as 
he  would  from  the  cockpit  of  an  aircraft.  Oome-type  visual  simulation  systems  have  been  used  in 
situations  where  good  quality  detail  is  important  and  judgments  of  range  and  range-rate  are  important 
aspects  of  training.  Projected  imagery  displays  have  been  used  in  air  combat  maneuvering,  formation 
flight,  and  aircraft  carrier  landing  training  systems. 

The  development  of  computer  generated  visual  display  systems  has  introduced  a  new  area  of  tech¬ 
nology  into  aircrew  training  simulation.  Computer  generated  imagery  (CGI)  systems  involve  computerized 

Generation  of  a  visual  environment  through  the  use  of  lines  or  edges  presented  on  a  cathode  ray  tube 
CRT)  or  by  discrete  point  light  source  elements  controlled  to  form  the  in-flight  visual  environment 
desired  to  support  instruction  and  practice  of  a  theoretically  limitless  variety  of  training  tasks. 
Dynamic  changes  in  the  CGI  display  during  a  training  exercise  are  controlled  by  the  simulator's  central 
digital  computer. 

The  visual  environments  produced  by  CGI  systems  may  be  modified  or  replaced  by  other  visual 
environments  without  acquiring  additional  equipment.  Once  developed,  the  visual  environment  models  may 
be  stored  and  used  again  rapidly  and  easily.  Because  of  this  capability,  CGI  systems  are  not  limited  in 
content  or  extent  and  represent  a  substantial  increase  in  performance  capability  over  the  older  TV-model 
and  projected  imagery  simulator  visual  display  systems. 

The  next  chapter  will  present  a  comprehensive  detailed  review  of  research  data  describing  flight 
simulator  training  effectiveness. 


V.  TRAINING  EFFECTIVENESS  OF  FLIGHT  SIMULATORS 


DETERMINING  THE  SIMULATOR  TRAINING  EFFECTIVENESS 

Advances  in  flight  simulation  technology  make  available  a  wide  variety  of  sophisticated  systems  and 
subsystems  for  combination  into  a  training  device  that  best  meets  the  demands  of  a  training  manager. 
Many  of  the  options  are  designed  to  increase  the  training  value  of  a  device  by  making  it  possible  to 
implement  innovative  instructional  and  training  methods.  The  capability  for  real-time  automated 
performance  measurement  and  feedback,  adaptive  training,  programed  demonstrations  and  review  by  the 
pilot  are  examples  of  training-oriented  features.  Other  options  currently  available  are  designed  to 
increase  training  effectiveness  by  increasing  the  performance  capabilities  of  the  device.  Wide 
field-of-view  visual  systems  of  a  variety  of  types,  synergistic  six  degree-of-freedom  platform  motion 
systems,  g-seats,  and  g-suits  are  typical  of  fidelity-oriented  hardware. 

The  training  manager  now  has  to  decide  how  many  of  these  features  are  necessary  for  the  intended  use 
of  the  device.  To  make  such  decisions,  he  must  define  the  training  objectives  and  estimate  how  much  the 
various  options  can  contribute  to  achieving  them.  He  must  also  determine  the  value  of  the  expected 
benefits  relative  to  the  cost  of  the  hardware  capability  required  to  yield  these  benefits. 
Unfortunately,  the  training  manager  is  often  in  the  position  of  having  to  make  such  decisions  without 
sufficient  information. 

Behavioral  research  can  provide  information  relative  to  several  important  criteria:  (1)  user 
acceptance;  (2)  the  feasibility  of  training  tasks  which  cannot  be  practiced  in  the  aircraft  (e.g.,  some 
emergency  situations,  missile  evasion  techniques);  and  (3)  training  effectiveness.  Evaluating  the 
training  effectiveness  of  a  device  is  one  of  the  most  important  types  of  information  for  the  training 
manager.  Military  training  managers  have  indicated  they  are  primarily  Interested  in  this  type  of 
research  results  (30).  Unfortunately,  suitable  training  effectiveness  data  can  be  time-consuming  and 
difficult  to  generate.  Recently,  Caro  (31)  has  summarized  methods  of  evaluating  simulator  training 
effectiveness.  Of  those  procedures,  he  indicated  that  the  transfer  of  training  methodology  is  "most 
appropriate  to  determine  whether  simulator  training  has  Improved  subsequent  operational  performance." 

In  the  transfer  study  design,  preliminary  training  Is  given  in  the  simulator  (pretraining)  followed 
by  a  conparative  performance  evaluation  In  the  aircraft  (criterion  system).  In  most  cases,  one  or  more 
experimental  treatments  are  compared  with  some  standard  (control)  treatment.  For  example,  a  conparison 
of  the  relative  training  effectiveness  of  two  visual  systems  would  require  three  groups — one  trained 
with  visual  system  A,  a  second  trained  with  visual  system  B,  and  a  third  receiving  no  simulation  pre¬ 
training.  .A  comparison  of  subsequent  performance  in  the  aircraft  between  groups  one  and  two  provides  an 
estimate  of  the  relative  effectiveness  of  visual  systems  A  and  B.  Comparing  the  combined  performance 
data  of  the  first  two  groups  with  the  performance  of  the  third  group  (control)  provides  an  estimate  of 
the  overall  effectiveness  of  the  simulation  training.  The  demonstration  of  effective  transfer  of 
training  Is  a  prerequisite  for  making  any  definitive  statements  concerning  the  relative  effectiveness  of 
alternate  systems. 
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In  the  following  paragraphs,  the  training  effectiveness  literature  will  be  reviewed  with  respect  to 
simulator  motion  ana  visual  systems.  The  addition  of  either  or  both  of  these  systems  adds  significantly 
to  procurement  as  well  as  operations/maintenance  costs.  For  this  reason,  it  is  necessary  to  Insure  that 
such  added  costs  are  justified  in  terms  of  an  inproved  training  capability  which  is  evidenced  by  enhanced 
piloting  skills  in  the  aircraft.  The  present  review  will  focus  on  data  obtained  through  the  application 
of  the  transfer  of  training  methodology,  since  such  Information  seems  most  relevant  to  this  issue. 

VISUAL  SIMULATION 

The  technology  of  visual  simulation  is  expanding  rapidly.  Most  flight  simulators  are  being  procured 
with  visual  systems.  Many  older  instrument  flight  simulators  are  being  retrofitted  with  some  type  of 
visual  capability.  Current  visual  simulation  systems  can  cost  up  to  six  million  dollars.  Despite  its 
cost,  the  potential  value  of  a  visual  system  is  great  since  it  presents  the  opportunity  to  train  tasks 
which  otherwise  would  have  to  be  learned  in  the  aircraft.  Furthermore,  it  offers  the  possibility  of 
substantial  cost  savings,  especially  for  those  aircraft  which  have  high  operating  costs.  Such  potential 
is  witnessed  by  recent  attenpts  to  extend  visual  simulation  training  into  such  areas  as  air  combat, 
weapons  delivery,  and  aerial  refueling.  Most  studies  to  date  have  focused  on  visual  simulation  training 
for  fixed  wing  aircraft.  Because  of  the  relatively  large  number  of  studies  and  diverse  missions  which 
are  simulated,  they  are  presented  according  to  task  categories.  Finally,  the  value  of  visual  simulation 
for  rotary  wing  training  will  be  addressed. 

Transition 

The  acquisition  of  basic  contact  skills  Including  takeoffs  and  landings  has  been  studied  most 
frequently.  The  first  series  of  controlled  studies  was  accomplished  at  the  University  of  Illinois' 
Institute  of  Aviation.  Williams  and  Flexman  (32)  taught  basic  aircraft  control,  stalls  and  traffic 
pattern  skills  in  a  l-CA-2  Link  trainer  simulating  the  SNJ  aircraft.  The  visual  scene  consisted  of  a 
270°  circular  screen  (cyclorama)  eleven  feet  high  placed  seven  feet  from  the  trainer.  The  screen  was 
an  unmarked  white  cloth  with  the  exception  of  a  black  horizontal  line  representing  the  horizon  and 
several  reference  marks  indicating  climb/descent  attitude  and  heading.  No  takeoff /landing  simulation 
was  provided.  Two  groups  of  24  students  participated  in  the  study.  The  experimental  group  received 
simulator  training  prior  to  aircraft  training  while  the  control  group  received  only  the  aircraft 
training.  The  simulator-trained  group  required  62*  fewer  trials  to  reach  proficiency,  committed  75* 
fewer  errors,  and  required  62*  less  flight  time. 

In  a  later  study,  Flexman,  Matheny  and  Brown  (33)  attempted  to  determine  whether  training  in  the 
l-CA-2  SNJ  Link  would  enable  students  to  pass  their  flight  check  with  only  10  hours  (as  compared  to  the 
normal  35)  of  aircraft  instruction.  In  this  effort,  contact  training  was  provided  for  takeoffs  and 
landings.  The  visual  landing  scene  consisted  of  a  blackboard,  placed  in  front  of  the  trainer,  which 
could  be  rotated  about  its  horizontal  axis.  A  rough  perspective  view  of  a  runway  was  drawn  on  the 
blackboard.  At  the  beginning  of  a  landing,  the  instructor  held  the  blackboard  approximately  at  a  45° 
angle.  The  instructor  then  gradually  reduced  the  angle  to  simulate  the  approach  to  the  runway.  As  the 
blackboard  approached  the  horizontal  plane,  the  trainer  appeared  to  be  near  the  ground.  Compared  to 
students  not  trained  this  way,  students  receiving  simulator  pretraining  performed  significantly  better 
in  that:  (1)  a  higher  percentage  passed  the  flight  check;  (2)  checkride  scores  were  higher;  and  (3) 
fewer  students  failed  four  or  more  flight  check  items.  No  direct  assessment  of  the  value  of 
approach/ landing  training  was  made.  The  value  of  such  training  for  aircraft  landing  performance  was 
specifically  assessed  in  a  follow-on  study  (34).  The  results  indicated  that  students  in  the 
experimental  group  (N=10)  following  three  hours  of  approach/landing  simulator  pretraining  committed 
significantly  fewer  errors  in  15  aircraft  landings  than  did  the  control  group  (N-10)  which  received  no 
simulator  pretraining.  Such  data  demonstrate  that  positive  transfer  effects  are  possible,  even  with 
very  crude,  low  fidelity  training  devices. 

Oespite  the  demonstrated  value  of  the  ‘blackboard"  visual  scene,  it  was  obvious  that  other  essential 
cues  were  missing.  Based  on  an  analysis  of  runway  perspective  by  Bell  (35),  an  experimental  landing 
display  projector  was  developed  for  use  on  the  l-CA-2  SNJ  Link  trainer.  The  runway  image  was  controlled 
by  heading  and  altitude  information  from  the  l-CA-2  and  was  displayed  on  a  screen  located  in  front  of 
the  trainer.  Payne,  Dougherty,  Hasler,  Skeen,  Brown  and  Williams  (36)  evaluted  the  effectiveness  of 
this  device  for  training  the  final  approach  to  landing.  Students  in  the  experimental  group  (N*6) 
received  simulator  pretraining  until  proficiency  criteria  were  reached.  Both  the  experimental  group  and 
control  group  (who  received  no  simulator  pretraining)  were  trained  to  the  same  proficiency  criteria  in 
the  SNJ  aircraft.  The  following  savings  were  obtained:  (1)  number  of  trials  to  reach  proficiency  -  61*; 
(2)  number  of  errors  to  reach  proficiency  -  74*;  (3)  number  of  errors  per  trial  -  50*;  (4)  number  of 
errors  on  the  first  trial  -  67*;  and  (5)  number  of  errors  on  the  first  five  trials  -  55*.  Analysis  of 
actual  landing  (touchdown)  data  which  was  not  trained  In  the  simulator  also  revealed  significant  savings. 

In  1953,  the  US  Air  Force  accepted  delivery  of  the  PI  simulator,  essentially  the  same  device 
(l-CA-2)  used  at  the  University  of  Illinois  In  earlier  studies.  In  order  to  evaluate  the  effectiveness 
of  the  device  for  contact  training,  95  aviation  cadets  were  divided  into  two  groups  (37).  The 
experimental  group  received  40  hours  of  simulator  training  and  100  hours  of  T-6  (SNJ)  aircraft 
instruction.  The  control  group  received  130  hours  of  T-6  aircraft  instruction  with  no  simulator 
training.  At  the  end  of  training,  both  groups  were  evaluated  according  to  certain  criteria.  The 
results  indicated:  (1)  significantly  better  flying  performance  of  the  simulator-trained  group  as 
measured  by  Oally  Progress  Record  Sheets;  (2)  significantly  better  checkride  scores  of  the 
simulator-trained  group  using  Independent  check  pilots;  and  (3)  no  differences  as  Indicated  by  a 
research-type  flight  check,  attrition  data,  or  accidents.  Ninety-two  percent  of  the  flight  Instructors 
felt  that  the  simulator-trained  students  were  "equal  to"  or  "better  than"  the  control  group  in  terms  of 
overall  proficiency. 
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Several  studies  at  the  US  Naval  School  of  Aviation  Medicine  also  evaluated  the  effectiveness  of  the 
SNJ  Link  trainer.  Poe  and  Lyon  (38)  provided  instruction  In  the  SNJ  Link  during  Preflight  School. 
Eighty-five  cadets  received  five  hours  of  training  in  the  device.  The  performance  of  this  group  during 
the  initial  stages  of  flight  training  was  compared  with  a  control  group  of  100  cadets  who  did  not 
receive  the  simulator  pretraining,  criteria  included  attrition  data,  flight  efficiency  data,  extra 
rides  required,  instructional  flight  grades,  and  checkride  scores.  No  statistical  differences  in 
performance  between  the  two  groups  were  found.  Creelman  (39)  reported  that  students  trained  In  the  SNJ 
Link  with  a  contact  landing  display  performed  significantly  better  than  students  who  received  either  no 
pretraining  or  simply  viewed  films  of  contact  landings.  The  simulator  trained  group  received  higher 
performance  ratings  on  their  aircraft  approaches,  required  fewer  practice  landings  prior  to  solo  and 
received  fewer  unsatisfactory  flight  grades. 

The  results  of  these  studies  conducted  by  the  University  of  Illinois,  the  US  Air  Force  and  the  US 
Navy  conclusively  demonstrated  that  visual  simulation  training  produced  significant  transfer  to  subse¬ 
quent  performance  in  the  SNJ/T-6  aircraft.  Significant  transfer  was  shown  for  basic  contact  skills  and 
the  final  approach  to  a  landing.  Following  these  Initial  efforts,  the  authors  are  unaware  of  any  other 
similar  transfer  of  training  studies  accomplished  prior  to  the  establishment  of  the  Operational  Training 
Division  of  the  Air  Force  Human  Resources  Laboratory  in  1969.  In  1970,  the  Laboratory  received  the  T-4G 
flight  simulator,  an  updated  ME-1  trainer  which  simulates  the  T-37  aircraft,  the  Air  Force’s  primary  jet 
trainer.  It  consisted  of  a  T-4  cockpit  mounted  on  a  two  degree-of-freedom  platform  motion  system.  A 
Singer  SPO  Electronic  Perspective  Transformation  visual  system  was  attached  which  enabled  training  in 
normal  straight-in  approaches  from  four  miles  out,  touchdowns,  landing  rolls,  and  takeoffs.  The  visual 
field  of  view  was  44°  x  38°  and  the  image  was  provided  in  full  color  at  infinity. 

The  effectiveness  of  the  T-4G  for  providing  both  contact  and  instrument  training  was  evaluated  by 
Woodruff  and  Smith  (40).  Twenty-one  students  were  given  pretraining  In  the  T-4G  followed  by  an 
evaluation  of  their  subsequent  performance  in  the  T-37  aircraft.  Training  In  both  the  simulator  and 
aircraft  continued  until  proficiency  criteria  were  attained.  For  the  contact  phase,  the  simulator 
pretraining  respited  In  an  average  savings  of  three  hours  In  the  T-37  aircraft  or  approximately  10*. 
These  comparisons  were  against  the  length  of  the  normal  syllabus  being  used  at  that  time.  Mid-phase 
contact  checkride  scores  revealed  no  differences  when  compared  against  the  scores  of  other  students  not 
receiving  the  simulator  pretraining. 

In  1975,  the  Advanced  Simulator  for  Pilot  Training  (ASPT)  initiated  research  operations  at  the 
Operational  Training  Division.  The  ASPT  is  equipped  with  two  T-37  cockpits.  Each  cockpit  has  a  full 
field-of-view  visual  display  (+150°  horizontal  by  +110°,  -40°  vertical)  of  computer-generated 
imagery  (CGI),  a  six  degree-of-freedom  platform  motion  system,  and  a  sixteen  panel  pneumatic  g-seat  on 
the  left  seat  (student  position).  The  visual  system  uses  an  infinity  optic  display  with  the  exit  pupil 
located  at  the  student's  eye  position.  The  scene  is  projected  through  seven  37"  diameter  cathode  ray 
tubes.  A  complete  description  of  the  ASPT  may  be  found  in  a  report  by  Gum,  Albery  and  Basinger  (41). 

Upon  acceptance  of  the  device.  Woodruff,  Smith,  Fuller  and  Weyer  (42)  conducted  an  exploratory  study 
to  investigate  the  utility  of  the  ASPT  as  a  full  mission  simulator  in  the  basic  phase  of  Air  Force 
Undergraduate  Pilot  Training.  Training  In  the  ASPT  was  provided  for  Basic  Contact,  Advanced  Contact 
(Aerobatics),  Instrunents,  Navigation,  and  Formation  Flight.  Upon  completion  of  each  block  of  training 
in  the  ASPT,  the  student  proceeded  to  the  aircraft  for  additional  instruction.  Eight  students  received 
ASPT  pretraining  while  a  control  group  of  eight  students  did  not.  Proficiency  advancement  was  used  for 
all  instruction  in  both  the  simulator  and  aircraft.  The  resulting  aircraft  hours  savings  were  45*  for 
Basic  Contact;  4*  for  Advanced  Contact;  38*  for  Instruments;  13*  for  Navigation;  and  13*  for  Formation. 
For  the  ASPT-tralned  group,  T-37  contact  checkride  scores  were  significantly  higher.  This  effect 
persisted  into  the  T-38  training  phase  in  which  checkride  scores  were  again  significantly  higher  for  the 
simulator- trained  group. 

Subsequent  to  this  demonstration  of  the  training  effectiveness  of  the  ASPT,  a  number  of  studies  have 
been  accomplished  using  the  transfer  of  training  design  to  evaluate  alternative  hardware 
configurations.  The  first  study  addressed  the  contributions  of  platform  motion  cueing  to  the 
acquisition  of  basic  contact,  approach,  and  landing  skills  in  Air  Force  Undergraduate  Pilot  Training 
(UPT)  (43).  Twenty-four  preflight  UPT  students  with  no  previous  jet  piloting  experience  were  randomly 
assigned  to  one  of  three  treatment  groups  (n=8):  (a)  Motion;  (b)  No  Motion;  and  (c)  Control.  Those 
students  assigned  to  the  control  group  received  the  standard  syllabus  of  preflight  and  flightline 
instruction.  The  students  in  the  two  experimental  conditions  received  Identical  pretraining  in  the  ASPT 
with  the  exception  of  the  presence  or  absence  of  platform  motion  cueing.  The  g-seat  was  not  used. 

The  simulator  training  syllabus  consisted  of  ten  ASPT  sorties  covering  Instruction  on  a  large  mmiber 
of  basic  contact  maneuvers,  including  basic  alrwork  (turns,  climbs,  etc.),  slow  flight,  stalls, 
takeoffs,  straight-in  approach  and  landing,  the  overhead  pattern,  and  the  touch-and-go.  Following 
simulator  pretraining,  the  students  were  evaluated  on  two  special  aircraft  sorties  by  research 
instructor  pilots  (IPs)  as  well  as  on  all  sorties  prior  to  solo  by  their  normal  flightline  IPs.  The 
control  group  did  not  receive  the  special  data  rides  for  safety  considerations.  It  was  observed, 
however,  that  a  number  of  the  experimental  students  were  able  to  perform  takeoffs  and  overhead 
approaches  and  landings  on  their  first  aircraft  ride.  An  analysis  of  the  data  collected  by  flightline 
IPs  revealed  significantly  better  performance  by  the  ASPT-tralned  groups  for  all  tasks  evaluated.  The 
percent  savings  in  terms  of  trials  rated  Unsatisfactory  were  51*  for  takeoff;  48*  for  straight-in 
approach;  33*  for  straight-in  landing;  42*  for  overhead  pattern;  37*  for  overhead  landing;  77*  for  slow 
flight;  61*  for  power-on  stalls;  and  55*  for  traffic  pattern  stalls. 

While  there  is  evidence  that  positive  transfer  occurs  for  even  the  crudest  of  visual  scenes,  there 
is  little  data  comparing  the  relative  effectiveness  of  alternative  approaches  to  visual  simulation. 
Martin  and  Cataneo  (44)  compared  the  effectiveness  of  ASPT  training  using  a  night  scene  vs.  a  day 
scene.  The  night  scene  was  modeled  to  closely  approximate  commercially  available  point  light  source  CGI 
visual  systems.  A  generalized  airport  scene  was  modeled  for  both  the  night  and  day  scenes,  so  that  the 
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simulator  was  not  specific  to  Williams  AFB.  Twenty-four  UPT  students  were  divided  into  three  groups 
(n=8):  (a)  Day;  (b)  Night;  and  (c)  Control.  The  day  and  night  groups  received  three  ASPT  training 
sorties  in  which  instruction  was  provided  on  the  takeoff,  straight- in  approach  and  landing,  and  the 
touch-and-go.  The  control  group  received  no  ASPT  pretraining.  Following  simulator  pretraining, 
students  (including  the  controls)  were  evaluated  on  their  second  and  fifth  aircraft  sorties  by  their 
flightline  IPs.  The  data  revealed  signficant  transfer  for  the  ASPT-trained  students  but  no  differences 
between  the  day  and  night  groups. 

Nataupsky,  Waag,  Weyer,  McFadden  and  McDowell  (45)  conpleted  a  study  to  determine  the  interaction  of 
motion  and  f ield-of-view  (FOV)  on  the  acquisition  of  transition  skills.  Four  groups  of  eight  novice  UPT 
students  were  trained  in  the  ASPT  under  the  following  conditions:  (1)  platform  motion,  full  FOV  (300° 
x  150°);  (2)  no  platform  motion,  full  FOV;  (3)  platform  motion,  limited  FOV  (48°  x  36°);  and  (4) 
no  platform  motion,  limited  FOV.  Each  student  received  four  ASPT  sorties  in  which  the  takeoff,  steep 
turn,  slow  flight,  and  straight-in  approach  and  landing  were  instructed.  Following  ASPT  instruction, 
each  student  was  evaluated  on  his  first  T -37  aircraft  ride.  Due  to  safety  considerations,  a  control 
group  was  not  possible.  Neither  motion,  f ield-of-view,  nor  their  interaction  impacted  subsequent 
performance  in  the  aircraft. 

For  training-type  aircraft,  it  is  clear  that  visual  simulation  training  aids  the  student  in 
effectively  transitioning  into  the  airborne  environment.  For  large  transport  aircraft,  the  results  are 
more  dramatic,  especially  within  the  airline  industry.  With  the  introduction  of  flight  simulators, 
American  Airlines  (46)  successfully  reduced  flying  time  for  their  Captain  upgrade  program  from  18.3  to 
1.3  hours  for  the  Boeing  707  and  from  20.6  to  1.0  hours  for  the  Boeing  727.  However,  it  must  be 
recalled  that  these  are  highly  experienced  pilots  who  already  have  a  great  deal  of  flight  time. 

\ 

In  1976,  the  US  Navy  accepted  delivery  of  Device  2F87F,  an  operational  flight  trainer  for  the 
P-3(C),  a  four-engine,  turbo-prop  aircraft.  The  2F87F  is  a  high  fidelity  device  equipped  with  a  six 
degree-of-freedom  platform  motion  system  and  TV  model  board  visual  system  with  a  50°  x  38°  f ield-of- 
view.  Browning,  Ryan,  Scott  and  Smode  (47)  completed  an  evaluation  of  its  training  effectiveness  in 
which  the  contribution  of  its  visual  system  was  one  of  the  primary  considerations.  An  experimental 
group  (n=27)  received  six  sorties  in  the  2F87F  followed  by  four  P-3  sorties.  The  controls  (n=74) 
received  three  sorties  in  Device  2F69D  (the  old  simulator  with  no  visual  system)  followed  by  six  P-3 
sorties.  Aircraft  hours  were  reduced  from  15  for  the  control  groups  to  8.6  for  the  experimental  groups. 
No  differences  were  obtained  for  average  checkflight  grades.  The  average  number  of  landings  in  the  air¬ 
craft  to  become  proficient  was  reduced  31*  from  52  to  36.  Furthermore,  experimental  students  committed 
significantly  fewer  errors  per  landing  than  the  control  group.  The  errors  per  landing  for  the  experi¬ 
mental  group  on  their  fourth  P-3  sortie  were  less  than  those  of  the  control  group  on  their  sixth  sortie. 

In  a  follow-on  effort,  Browning,  Ryan  and  Scott  (48)  collected  additional  data  for  a  group  of  pilots 
(n= 10)  who  received  aircraft  training  only--that  is  no  simulator  pretraining  with  either  device  2F69D  or 
device  2F87F.  The  average  number  of  aircraft  hours  required  for  proficiency  was  15.1,  the  same  number 
for  those  students  (n=58)  receiving  training  in  the  device  2F69D,  the  old  operational  flight  trainer. 
This  conpared  to  only  8.6  hours  required  by  the  group  (n*27)  receiving  training  in  the  2F87F.  The 
number  of  aircraft  landings  required  for  proficiency  was  17  for  the  2F87F-trained  group  as  compared  with 
50  for  the,  aircraft-only  group.  It  was  also  reported  that  students  trained  to  proficiency  in  the 
simulator  had  a  higher  probability  of  demonstrating  proficiency  in  the  aircraft  on  earlier  flights  than 
students  not  trained  to  proficiency.  These  data  clearly  demonstrate  the  effectiveness  of  visual 
simulation  training  for  large  transport-type  aircraft. 

Thorpe,  Varney,  McFadden,  LeMaster  and  Short  (49)  reported  a  transfer  of  training  study  designed  to 
determine  the  relative  training  effectiveness  of  three  visual  systems:  a  Day/Night  Color  CGI  system;  a 
Night-Only  Point  Light  Source  CGI;  and  a  TV/Model  Board  system.  For  convenience,  they  are  designated 
Day,  Night  and  TV.  Thirty  recent  UPT  graduates  transitioning  into  the  copilot  position  of  the  KC-135 
(an  in-flight  refueling  aircraft)  were  given  training  on  the  visual  traffic  pattern,  approach  and 
landing.  These  subjects  were  divided  into  three  equal  groups,  each  receiving  simulator  training  using 
one  of  the  three  visual  systems.  Training  was  accoityl ished  in  Boeing  707  commercial  flight  simulators 
rented  from  the  Boeing  Aerospace  Company  (Day  system)  and  the  American  Airlines  Flight  Academy  (Night 
and  TV  systems).  Each  student  received  up  to  a  maximum  of  eight  hours  of  training  in  the  simulator  with 
Instruction  provided  by  KC-135  Instructor  pilots.  Following  instruction  in  the  simulator,  each  student 
flew  two  sorties  In  the  KC-135  aircraft.  On  each  sortie,  the  student  flew  three  to  four  repet It-it ions 
of  the  approach  and  landing.  Upon  completion  of  the  two  evaluation  sorties,  each  student  entered  the 
normal  KC-135  copilot  training  program.  The  final  evaluations  each  student  received  at  the  end  of 
training  were  recorded. 

Analysis  of  student  performance  on  the  two  aircraft  evaluation  sorties  revealed  a  statistically 
reliable  difference  between  the  TV  group  and  the  two  CGI  groups.  The  Night  and  Day  (CGI)  groups 
performed  significantly  better  than  the  TV  group.  No  differences  were  found  between  the  Day  and  Night 
groups.  These  differences  were  observered  on  the  last  two  segnents  of  the  task,  the  final  approach  and 
landing.  The  data  revealed  that  the  Day  and  Night  groups  Improved  their  performance  from  the  first  to 
the  second  evaluation  sortie.  The  TV  group,  however,  revealed  no  Improvement.  The  major  areas  of 
weakness  for  the  TV  group  were  in  the  glidepath  and  landing  segments  of  the  task,  with  substantially 
more  extreme  deviations  in  the  latter  stages  of  the  glidepath.  Such  trends  were  not  evident  in  the 
performance  of  the  Day  and  Night  groups. 

Resources  did  not  permit  the  incorporation  of  a  true  control  group  In  the  design  of  the  Thorpe  et 
al.  (49)  study;  that  Is,  a  group  receiving  only  the  two  aircraft  evaluation  sorties  with  no  simulator 
prebraining.  However,  to  obtain  an  estimate  of  the  effectiveness  of  simulator  training,  the  final 
checkride  scores  of  students  participating  In  the  study  were  compared  with  those  of  students  In  previous 
and  subsequent  classes.  Reliable  differences  were  obtained  with  60X  of  the  simulator- trained  students 
receiving  a  "Highly  Qualified"  evaluation  compared  with  only  30$  of  normal  students  (non- simulator 
trained)  receiving  this  score.  This  finding  was  further  supported  by  the  judgment  of  experienced 
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instructors  who  felt  tne  simulator- trained  students  Initially  performed  at  a  skill  level  comparable  to 
the  average  student  copilot  well  along  In  the  training  program. 

For  fighter-attack  aircraft,  little  Information  is  available  regarding  the  effectiveness  of  visual 
simulation  training  for  transition  tasks.  Brlctson  and  Burger  (50)  report  an  evaluation  of  Device 
2F103,  a  night  carrier  landing  trainer  (NCLT)  for  the  A-7E  aircraft.  Device  2F103  consists  of  an  A-7E 
cockpit  with  a  night  only,  point  light  source  CGI  visual  system  mounted  on  a  three  DOF  motion  system. 
The  visual  system  has  a  40°h  x  30°V  f ield-of-view  and  presents  a  color  image  of  the  deck  lighting 
and  visual  landing  aids  of  several  carrier  types.  A  syllabus  was  developed  consisting  of  6.5  hours 
which  enabled  about  85  simulated  night  carrier  landings  to  be  accomplished.  The  experimental  group, 
consisting  of  26  novice  pilots,  received  training  In  the  device  while  a  control  group  (n*27)  did  not. 
Performance  during  Field  Carrier  Landing  Practice  (FCLP)  and  Carrier  Qualification  (CQ)  clearly 
demonstrated  the  effectiveness  of  the  simulation  training  as  measured  by  an  objective  landing 
performance  score  and  boarding  rate.  For  the  experimental  group,  only  one  student  failed  CQ  compared 
with  seven  for  the  control  group. 

Since  failure  leads  to  recycling  in  which  the  student  drops  back  to  the  next  class,  the  question  of 
the  use  of  the  NCLT  for  remedial  training  was  raised.  Brictson  (51)  developed  a  technique  for 
identifying  students  in  need  of  remedial  training  and  also  a  syllabus  of  instruction  using  the  NCLT.  In 
an  experimental  evaluation,  students  trained  with  this  syllabus  received  higher  scores  during  FCLP  and 
CQ.  Furthermore,  their  boarding  rate  (successful  approach  and  touchdown)  was  higher  than  that  of  groups 
receiving  the  normal  NCLT  syllabus  of  instruction.  The  data  from  these  two  studies  clearly  demonstrate 
the  effectiveness  of  visual  simulation  training  for  night  carrier  landings. 

Gray,  Hamer,  Eubanks  and  Chun  (52)  recently  completed  a  study  to  determine  the  effectiveness  of 
ASPT  training  for  students  transitioning  Into  the  A-10.  The  ASPT,  originally  a  T-37  simulator,  was 
modified  to  an  A-10  configuration  for  training  transition  and  surface  attack  skills.  On  their  first 
overhead  pattern  sortie,  students  trained  in  this  simulator  demonstrated  proficiency  enabling  them  to 
land  out  of  their  fifth  pattern.  Experienced  fighter  pilots  transitioning  Into  the  A-10,  however,  were 
landing  out  of  their  eighth  pattern.  Although  scores  from  this  do  not  represent  true  control  group 
data,  they  do  illustrate  the  effectiveness  of  the  training. 

Formation 

The  acquisition  of  skill  in  formation  flying  Is  one  of  the  more  critical  and  demanding  tasks  in 
military  aviation.  At  present  there  are  only  a  few  devices  which  can  provide  such  training.  A 
simplified  formation  flight  trainer  (FFT)  was  developed  for  the  US  Air  Force  Human  Resources  Laboratory 
in  the  early  1970s.  It  was  designed  as  a  part-task  trainer  which  would  provide  closed-loop  practice  for 
nine  formation  tasks  learned  during  the  T-38  phase  of  UPT.  The  device  enabled  the  student  to  "fly"  a  TV 
camera  which  viewed  a  model  of  the  simulated  lead  aircraft.  The  resulting  image  of  the  lead  aircraft 
was  projected  onto  a  wide  screen  which  the  student  viewed  from  a  simplified  T-38  cockpit.  Horizon  and 
cloud  cover  Imagery  could  be  provided  by  a  prograirmed  point  light  source  projection  of  a  spherical 
transparency.  A  detailed  description  of  the  device  can  be  found  in  Wood,  Hagin,  O'Connor  and  Myers  (53). 

The  effectiveness  of  the  FFT  was  evaluated  by  Reid  and  Cyrus  In  two  separate  studies.  In  Study  I 
(26),  70  UPT  students  in  the  T-38  phase  of  training  were  randomly  assigned  to  one  of  three  groups.  The 
FFT  group  received  five  sorties  of  Instruction  In  the  FFT,  an  orientation  ride  In  the  T-38  and  finally  a 
checkride  in  the  T-38.  A  Limited  Training  group  received  only  an  orientation  ride  followed  by  the 
checkrlde.  The  UPT  Syllabus  group  received  two  aircraft  training  sorties  between  their  orientation  ride 
and  checkrlde.  Results  from  the  checkrlde  Indicated  both  the  FFT  and  UPT  Syllabus  groups  performed 
significantly  better  than  the  Limited  Training  group.  However,  no  differences  were  observed  between  the 
FFT  and  UPT  Syllabus  groups.  In  other  words,  five  hours  of  FFT  Instruction  were  as  effective  as  two 
hours  of  aircraft  instruction.  The  same  design  was  used  In  a  second  study.  The  only  difference  was 
that  an  Air  Training  Conmand  syllabus  change  had  occurred  In  which  students  were  given  additional 
formation  training  during  the  T-37  phase.  Using  48  students,  the  study  was  replicated.  The  results 
were  the  same  thereby  providing  conclusive  evidence  that  the  FFT  was  an  effective  trainer. 

In  a  follow-on  study  (54),  the  same  design  was  used  to  determine  effectiveness  of  the  FFT  for  the 
T-37  phase  of  UPT.  The  FFT  was  modified  to  provide  a  T-37  visual  Image  and  its  flight  dynamics  were 
changed  to  approximate  those  of  the  T-37,  although  the  cockpit  and  controls  remained  the  same.  A  total 
of  61  student  pilots  participated  in  the  study.  The  results  Indicated  that  the  UPT  syllabus  group 
performed  significantly  better  on  their  checkrlde  than  either  the  FFT  or  Limited  Training  groups. 
Although  the  FFT  group  had  higher  scores  than  the  Limited  Training  group,  the  difference  was  not 
statistically  significant.  The  extent  to  which  these  results  are  due  to  the  degraded  fidelity  of  the 
device  1$  unknown. 

The  only  other  effort  to  evaluate  formation  training  In  a  simulator  was  an  effort  by  Woodruff  et  al. 
(42),  described  previously,  using  the  ASPT.  In  that  study,  the  data  revealed  a  savings  of  13X  and  a 
transfer  Effectiveness  Ratio  (TER)4  of  1.00.  At  the  time  of  the  study,  a  number  of  equipment  problems 
were  encountered  which  led  to  the  decision  to  limit  the  formation  training  to  only  two  sorties. 
Furthermore,  for  three  of  the  eight  students,  these  sorties  were  cancelled  due  to  scheduling  conflicts, 
so  that  the  results  were  based  on  only  five  pilots.  However,  the  high  TER  Indicates  that  substantial 
savings  may  have  been  possible  If  additional  sorties  had  been  given. 


*  The  TER  Is  calculated  by  dividing  the  difference  between  the  aircraft  training  time  of  the 
experimental  and  control  groups  by  the  difference  between  the  simulator  training  time  of  the 
experimental  and  control  groups  (55).  It  provides  a  measure  of  the  relative  effectiveness  of 
training  which  can  be  compared  across  different  simulators  and  training  programs. 


Aerobatics  and  Air  Combat  Maneuvering 


The  ability  to  provide  training  for  aerobatics  and  air  combat  maneuvering  has  been  made  possible  by 
the  development  of  wide  angle  visual  systems.  Two  studies  were  completed  in  the  ASPT  in  which  there  was 
an  attempt  to  train  aerobatic  skills.  The  first,  reported  by  Woodruff  et  al.  (42),  revealed  that  6.2 
hours  of  instruction  in  the  ASPT  resulted  in  only  a  4*  savings  of  aircraft  time.  Recently,  Martin  and 
Waag  (56)  reported  an  effort  to  determine  the  contribution  of  platform  motion  to  the  acquisition  of 
aerobatic  skills.  Thirty-six  UPT  students  were  assigned  to  one  of  three  treatment  groups  (n=12):  (1) 
Motion;  (2)  No  Motion;  and  (3)  Control.  Students  in  the  two  experimental  groups  received  five  ASPT 
sorties  covering  instruction  on  eight  aerobatic  tasks.  The  control  group  did  not  receive  ASPT 
pretraining.  All  students  were  subsequently  evaluated  in  the  T-37  aircraft  by  their  normal  flightline 
IPs.  The  obtained  data  suggested  a  modest  degree  of  transfer.  Of  the  eight  maneuvers  trained  in  the 
ASPT,  only  one,  the  Barrel  Roll,  produced  an  overall  significant  transfer  effect  across  the  three 
groups.  However,  approximately  one  third  of  the  ASPT-trained  vs  Control  group  a  priori  t-tests  produced 
significant  effects.  In  all  cases,  superior  performance  was  demonstrated  by  the  ASPT-trained  groups. 
An  examination  of  group  means  indicated  the  trends  to  favor  the  simulator-trained  group  for  all  except 
three  of  the  measures  taken.  From  these  data,  it  is  apparent  that  transfer  of  training  did  occur. 
However,  the  magnitude  of  the  effect  was  not  great. 

Payne,  Hirsch,  Semple,  Farmer,  Sanders,  Wimer,  Carter  and  Hu  (57)  reported  a  study  to  determine  the 
amount  of  transfer  that  can  be  obtained  through  simulation  training  of  visual  air  combat  tasks. 
Subjects  were  16  Navy  pilots  transitioning  into  the  F-4.  The  eight  pilots  comprising  the  experimental 
group  received  six  training  sorties  in  the  Northrup  Large  Amplitude  Simulator/Wide  Angle  Visual  System 
(LAS/WAVS).  The  LAS/VttVS  has  a  spherical,  wide  angle  screen  which  provides  a  2100  horizontal 
f ield-of-view.  A  maneuverable  adversary  aircraft  as  well  as  an  earth-sky  image  is  projected  onto  the 
screen.  Training  was  provided  for  basic  fighter  maneuvers  such  as  barrel  roll  attacks,  high  yo-yos, 
rolling  scissors,  etc.  All  students  were  subsequently  evaluated  during  their  normal  tactics  syllabus 
which  consisted  of  six  sorties.  Analysis  of  data  reflecting  final  position  outcome  revealed  the 
experimental  group  achieved  superior  final  positions  when  compared  with  the  control  group.  This  held 
for  starts  at  the  neutral  as  well  as  offensive  positions.  Transfer  estimates  based  on  such  outcomes 
ranged  from  26*  to  96*.  The  superiority  of  the  experimental ly- trained  group  also  was  reflected  in  the 
grades  assigned  by  the  instructors.  These  differences  were  maintained  throughout  the  entire  tactics 
syllabus  of  instruction.  The  greatest  transfer  effects  were  demonstrated  for  the  rolling  scissors. 
Although  the  experimental  group  had  higher  scores  for  all  tasks,  the  differences  were  not  significantly 
higher  than  the  control  group. 

Pohlmann  and  Reed  (58)  completed  a  study  designed  to  determine  the  contribution  of  platform  motion 
to  the  initial  acquisition  of  basic  fighter  maneuver  skills,  the  same  type  of  tasks  studied  by  Payne  et 
al.  (57).  The  study  was  accomplished  on  the  Simulator  for  Air-to-Air  Combat  (SAAC),  a  device  comprised 
of  two  F-4  cockpits  each  mounted  on  a  synergistic  six  degree-of-freedom  motion  system.  The  visual 
display  consists  of  eight  pentagonal  cathode  ray  tubes  which  provide  a  2960h  x  150°V  fieid-of-view. 
A  camera  model  aircraft  image  generator  and  synthetic  terrain  generator  provided  the  images  for  the 
visual  display.  Sixteen  students  received  seven  training  sorties  in  the  SAAC.  All  students,  including 
six  control  students,  were  evaluated  in  subsequent  aircraft  sorties.  One  additional  aircraft  sortie  was 
added  to  the  normal  syllabus  to  assist  in  the  evaluation.  An  analysis  of  data  collected  in  the  aircraft 
revealed  no  enhancement  of  performance  as  a  result  of  simulator  pretraining.  In  fact,  the  trend  was 
toward  superior  performance  by  the  control  group. 

Air-To-Surface  Weapons  Delivery 

In  1975,  the  Air  Force  Simulator  Systems  Program  Office  Initiated  an  effort  to  evaluate  existing 
simulator  visual  system  technologies  that  were  applicable  to  air-to-surface  weapons  delivery.  Because 
of  its  CIS  capability,  one  of  the  systems  selected  for  consideration  was  the  ASPT.  A  new  environmental 
data  base  was  created  which  included  an  airfield  complex,  a  conventional  gunnery  range,  and  two  tactical 
gaming  areas.  Of  the  systems  evaluated,  the  ASPT  was  the  only  one  considered  capable  of  providing 
effective  air-to-surface  weapon  delivery  training  (59).  At  the  same  time,  the  Tactical  Air  Command 
requested  that  AFHRL  initiate  research  studies  to  demonstrate  the  training  value  of  platform  motion. 
Since  air-to-surface  weapons  delivery  was  one  of  the  task  areas  for  which  such  information  was  desired 
and  since  the  ASPT  was  the  only  system  considered  capable  of  training  such  tasks,  the  Operational 
Training  Division  initiated  a  study  to  determine:  (1)  the  extent  to  which  generalized,  conventional 
air-to-surface  weapons  delivery  training  in  the  ASPT  transferred  to  a  specific  aircraft;  and  (2)  the 
contribution  of  six  degree-of-freedom  platform  motion  to  the  transfer  of  training  from  simulator  to 
aircraft  (60).  Twenty-four  graduates  of  fighter  lead-in  training  were  assigned  to  one  of  three  treat¬ 
ment  groups  (n-8):  (a)  Motion;  (b)  No  Motion;  and  (c)  Control.  Simulator  pretraining  was  accomplished 
In  the  ASPT  which  simulates  the  T-37  aircraft  while  evaluations  were  conducted  in  the  F-5B  aircraft. 
Upon  arrival  at  Williams  AFB,  all  students  received  academic  training  In  weapons  delivery  techniques  and 
procedural  training  in  F-5B  operations.  At  this  point,  students  in  the  Control  group  flew  two  data 
collection  sorties  in  the  F-5B  aircraft,  performing  two  10°,  15°,  and  30°  bomb  deliveries.  At  the 
end  of  simulator  training,  students  in  the  experimental  groups  flew  the  same  two  evaluation  sorties  in 
the  F-58  aircraft. 

Four  sets  of  analyses  were  conducted  on  data  collected  in  the  aircraft.  Measures  included  the 
number  of  bombs  meeting  TAC's  qualifications  criteria,  the  number  of  bombs  which  were  scorable  on  the 
range,  circular  error,  and  IP  ratings.  The  two  simulator  groups  performed  significantly  better  than  the 
control  group  for  all  measures  except  the  IP  ratings.  The  two  experimental  groups  dropped  about  twice 
as  many  scorable  bombs  and  bombs  meeting  the  qualification  criteria  and  produced  an  average  circular 
error  of  about  25*  less. 

The  results  of  this  study  clearly  demonstrate  that  full  fidelity  simulation  is  not  necessary  for 
effective  transfer  of  training.  The  T-37  Is  the  primary  jet  trainer  used  In  the  Initial  stages  of  UPT 
while  the  F-58  Is  a  high  performance  fighter.  Prior  to  their  two  evaluation  sorties,  these  students  had 
never  flown  the  F-5,  although  they  had  flown  the  T-38  which  is  similar  to  the  F-5.  The  fact  that 
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generalized  training  in  a  T-37  simulator  transferred  to  the  F-5  questions  the  validity  of  the  design 
goal  of  maximum  fidelity. 

As  mentioned  previously,  the  modification  of  one  ASPT  cockpit  to  an  A-10  configuration  was  completed 
in  1977.  In  addition  to  transition  training,  recent  UPT  graduates  entering  the  A-10  training  program 
also  were  provided  weapons  delivery  training  on  the  ASPT  (52).  The  surface  attack  syllabus  consisted  of 
three  two-hour  sorties.  The  results  of  the  ASPT  training  for  the  first  17  pilots  to  complete  training 
in  the  ASPT  were  dramatic.  On  the  first  bombing  sorties  in  the  A-10,  the  average  circular  error  for  the 
30°  dive  bomb  event  was  substantially  less  than  the  TAC  criterion  for  qualification.  It  was  about  the 
same  as  the  average  circular  error  for  experienced  fighter  pilots  on  their  sixth  sortie.  A  later  class 
of  ASPT-trained  students  received  weapons  delivery  training  in  the  aircraft  first,  thereby  providing 
control  group  data.  Average  circular  error  on  the  first  sortie  for  the  30°  dive  bomb  event  was 
approximately  twice  that  of  students  receiving  pretraining  in  ASPT.  These  data  clearly  demonstrate  the 
effectiveness  of  the  training. 

Rotary  Wing  Studies 

In  1977,  the  US  Army  accepted  delivery  of  the  CH-47FS  operational  flight  simulator.  The  trainer  was 
designed  to  simulate  the  CH-47C  helicopter.  It  is  mounted  on  a  six  degree-of-freedom  platform  motion 
system  and  has  a  camera-model  visual  system  which  provides  a  48°H  x  36°V  f ield-of-view  display  in 
the  forward  window.  It  also  has  a  "chin  window"  display  which  utilizes  a  synthetic  terrain  generator. 
The  test  and  evaluation  of  the  device  incorporated  a  transfer  of  training  study  design.  Two  studies 
were  completed:  the  first,  assessed  the  effectiveness  of  the  CH-47FS  for  novice  pilots  transitioning 
into  the  CH-47  (61);  the  second,  assessed  the  effectiveness  of  the  CH-47FS  for  maintaining  mission 
readiness  skills  (62). 

For  the  initial  transfer  evaluation,  24  student  pilots  ware  trained  to  proficiency  in  the  CH-47FS. 
They  were  then  given  a  checkride  in  the  CH-47  aircraft,  followed  by  Instruction  on  those  tasks  beyond 
the  capabilities  of  the  CH-47FS  as  well  as  those  tasks  on  which  the  student's  performance  was  considered 
unsatisfactory.  At  the  end  of  training,  a  final  aircraft  checkride  was  administered.  The  control  group 
(n=35)  received  all  Instruction  in  the  aircraft  using  the  same  proficiency  advancement  and  checkride 
procedures.  Training  effectiveness  ratios  were  computed  on  the  basis  of  total  training  time  and 
training  trials  for  each  maneuver.  For  total  training  time  (exclusive  of  checkrides),  the  resulting  TER 
was  .72.  On  a  breakdown  by  maneuver,  TERs  ranged  from  .40  to  1.50  for  trials  to  criterion.  For  total 
time,  however,  the  TERs  ranged  from  -.43  to  1.69.  As  expected,  the  highest  TERs  were  found  for 
procedural  tasks  and  the  lowest  for  approaches  and  takeoffs.  An  evaluation  of  the  final  checkride 
scores  revealed  higher  scores  by  the  experimental  group  although  the  difference  was  not  statistically 
significant. 

In  the  second  study,  16  aviators  who  were  qualified  and  current  in  the  CH-47,  received  five  hours  of 
instruction  and  practice  in  the  CH-47FS  per  month  over  a  six-month  test  period.  Such  practice  was  in 
addition  to  their  mission  essential  flying  In  the  CH-47  aircraft.  A  control  group  of  16  aviators 
received  only  their  normal  mission  essential  flights  in  the  CH-47.  Checkrides  were  administered  at  the 
beginning  and  end  of  the  six-month  test  period  for  all  participants.  During  the  test  period,  there  were 
no  reliable  differences  between  the  groups  in  terms  of  mean  CH-47  aircraft  flight  time.  The  pretest 
checkride  indicated  significantly  better  performance  by  the  control  group.  The  post-test  checkride 
revealed  no  differences.  A  pretest/post- test  comparison  for  the  control  group  revealed  no  change  in 
performance.  For  the  experimental  group,  however,  there  occurred  a  significant  enhancement  of 
performance.  Of  the  35  individual  tasks  performed,  significant  improvement  was  observed  on  26.  The 
only  areas  not  showing  improvement  were  external  load  procedures  and  autorotations.  It  was  speculated 
that  this  may  have  been  due  to  limitations  of  the  visual  system. 

Summary 

Of  the  studies  completed  to  date,  most  have  focused  on  the  use  of  visual  simulation  for  transition 
training.  With  few  exceptions,  the  overwhelming  finding  is  that  visual  tasks  learned  In  the  simulator 
show  positive  transfer  to  the  aircraft.  The  successful  use  of  visual  simulation  training  has  been 
demonstrated  for  trainer,  fighter  and  transport  fixed-wing  aircraft  as  well  as  for  rotary  wing 
aircraft.  Such  effects  have  been  obtained  for  pilots  initially  transitioning  into  the  aircraft,  and  in 
one  Instance,  for  enhancing  the  skill  level  of  experienced  pilots  in  an  operational  flying  environment. 
Few  studies  have  been  accomplished  for  tasks  other  than  transition.  This  is  due  to  the  limited 
availability  to  date  of  the  wide-angle  visual  systems  needed  to  provide  suitable  training  on  contact 
tasks.  Nonetheless,  the  data  thus  far  suggests  that  significant  transfer  can  be  obtained  through 
simulator  training  of  skills  needed  for  formation  and  surface  attack  weapons  delivery.  Only  a  modest 
amount  of  transfer  has  been  demonstrated  for  aerobatic  and  air  combat  skills. 

NOTION  SIMULATION 

The  technology  of  motion  simulation  also  has  expanded  rapidly.  Today,  there  are  a  variety  of 
devices  which  can  provide  force  cueing  information.  These  include  platform  motion  systems,  g-seats, 
g-suits,  stick  shaker,  and  buffet/vibration  systems.  They  are  designed  to  provide  either  onset  or 
sustained  motion  cue  information.  Unlike  the  addition  of  a  visual  system,  force  cueing  devices  enable 
the  pilot  to  perform  only  a  few  additional  tasks  which  would  not  otherwise  be  learned  in  the  air.  In 
most  Instances,  force  cues  provide  only  secondary  Information  to  the  pilot.  In  Instrument  flight,  the 
pilot  is  trained  to  "fly"  only  by  instruments  and  to  ignore  force  cueing  information.  It  Is  well  known 
that  motion  cues  are  not  essential  for  effective  simulator  training  since  pilots  have  been  acquiring 
flying  skills  with  the  aid  of  fixed  base  devices  for  years.  However,  the  extent  to  which  these  recently 
developed  force  cueing  systems  add  to  the  effectiveness  of  simulation  training  in  terms  of  increased 
transfer  Is  unknown.  There  Is  evidence  that  single-axis  tracking  performance  is  enhanced  as  a  result  of 
simulated  motion  generated  In  a  laboratory  test  apparatus  (63).  Furthermore,  performance  in  the 
simulator  may  be  enhanced  under  certain  conditions.  However,  the  extent  to  which  this  additional  cueing 
In  simulators  enhances  the  training  performance  In  the  aircraft  has  only  recently  been  questioned. 
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Koonce  (64)  reported  a  study  which  investigated  the  effects  of  refresher  Instrument  training  in  a 
Singer-Link  GAT-2  on  subsequent  performance  in  a  Piper  Aztec.  The  GAT-2  is  mounted  on  a  limited 
two-and-a-half  degree-of-freedom  platform  motion  system.  Two  groups  were  trained  with  the  aid  of  motion 
cueing:  one  with  a  washout  drive  algorithm  and  the  other  with  sustained  drive  algorithm.  A  control 
group  was  trained  without  motion  cueing.  Each  pilot  received  two  simulator  sorties  followed  by  an 
aircraft  checkride.  During  two  simulator  sorties,  the  two  motion  groups  performed  significantly  better 
than  the  no  motion  group.  However,  in  the  aircraft  sortie,  the  no  motion  group  performed  better  than 
the  two  motion  groups,  although  the  differences  were  only  marginally  significant. 

In  a  follow-on  study,  Jacobs  (65)  trained  novice  students  in  the  GAT-2  and  subsequently  evaluated 
their  flight  performance  in  a  Piper  Cherokee  Arrow.  Thirty-six  students  were  divided  into  four  groups 
which  received:  (1)  simulator  training  with  normal  washout  motion;  (2)  simulator  training  with 
directionally  random  motion;  (3)  simulator  training  with  no  motion;  and  (4)  aircraft  training  only. 
Each  student  in  the  simulator-trained  groups  received  four  sorties  in  which  the  number  and  sequence  of 
task  repetitions  were  fixed.  Training  in  the  aircraft  was  accomplished  on  a  proficiency  basis.  Within 
the  simulator  trained  groups,  the  washout  motion  group  committed  significantly  fewer  errors  than  the  no 
motion  group.  Errors  in  the  random  washout  group  were  similar  to  those  of  the  no  motion  group. 
Analysis  of  the  aircraft  data  revealed:  (1)  significant  transfer  of  all  three  groups  in  terms  of  time  to 
criterion,  trials  to  criterion,  and  number  of  errors;  and  (2)  no  reliable  difference  among  the  three 
simulator  trained  groups.  The  results  of  these  two  studies  conducted  at  the  University  of  Illinois 
indicate  that  motion  cueing  did  not  substantially  enhance  the  transfer  of  training  to  the  aircraft. 

The  only  other  studies  of  platform  motion  effectiveness  have  been  conducted  by  the  Operational 
Training  Division  of  the  US  Air  Force  Human  Resources  Laboratory.  Most  have  been  described  in  the 
previous  section  concerning  the  effectiveness  of  visual  simulation;  for  these  cases,  only  the  findings 
pertinent  to  the  question  of  platform  motion  will  be  discussed.  Part  of  the  study  reported  by  Woodruff 
and  Smith  (40)  concerned  the  effectiveness  of  the  T-4G  simulator  for  instrument  training.  The  use  of 
this  device,  which  is  mounted  on  a  two  DOF  motion  base  and  has  a  limited  visual  system,  resulted  in  an 
average  reduction  of  10.1  flight  hours  in  the  T-37  aircraft.  Use  of  the  T-4,  which  is  the  same  trainer 
without  motion  or  visual  systems,  resulted  in  an  8.1  flight  hour  reduction.  The  difference  (10.1  vs  8.1 
hours),  however,  was  not  statistically  significant. 

In  the  exploratory  study  investigating  the  utility  of  the  ASPT  as  a  full  misson  simulator  in  the 
basic  phase  of  UPT  (42),  half  of  the  students  were  trained  with  platform  motion  (n=4)  and  the  other  half 
without  (n=4).  No  significant  differences  were  obtained  for  either  required  simulator  hours  or  required 
aircraft  hours.  This  finding  was  obtained  for  the  basic/presolo,  advanced  contact,  instruments  and 
navigation  phases  of  training. 

Following  final  acceptance  of  the  ASPT  in  1975,  an  unpublished  exploratory  study  was  conducted  which 
evaluated  the  contributions  of  platform  motion  to  the  acquisition  of  basic  contact  skills.  Two  groups 
(n=4)  were  trained  to  proficiency  in  the  simulator  and  Subsequently  evaluated  in  the  T-37  aircraft.  No 
differences  in  either  simulator  or  aircraft  performance  were  obtained.  In  a  subsequent  effort,  Martin 
and  Waag  (43)  addressed  the  same  question  using  more  rigorous  control  procedures  and  a  larger  sample 
size.  As  discussed  in  the  previous  section,  two  groups  of  students  (one  trained  with  motion,  the  other 
without),  received  ten  sorties  of  instruction  in  the  ASPT  on  basic  contact  skills.  Subsequent 
evaluations  in  the  T-37  aircraft  revealed  substantial  transfer  of  training.  However,  with  respect  to 
the  two  experimental  groups,  i.e..  Motion  and  No  Motion,  no  statistically  reliable  differences  were 
found  for  either  performance  in  the  simulator  or  subsequent  performance  in  the  aircraft.  In  the 
aircraft,  this  finding  was  observed  for  student  performance  on  two  special  data  sorties  at  the  beginning 
of  training,  and  for  their  performance  up  to  the  pre-solo  check. 

In  a  subsequent  study,  the  evaluation  of  platform  motion  effectiveness  was  extended  to  aerobatic 
skills  since  motion  cues  should  be  more  prominent  for  such  tasks  (56).  As  discussed  in  the  previous 
section,  the  data  revealed  only  a  modest  degree  of  transfer  to  the  aircraft.  A  comparison  between  the 
Motion  and  no  Motion  groups  revealed  some  small,  although  inconsistent  performance  differences  during 
simulator  training.  Of  those  individual  aircraft  measures  demonstrating  significantly  better 
performance  by  the  simulator  trained  groups  (13  of  40),  none  revealed  a  reliable  effect  due  to  motion. 

Since  both  of  these  studies  used  a  wide  field-of-vlew  (300°H  x  150°V),  it  was  speculated  that 
peripheral  visual  cues  may  be  imparting  "motion"  cues.  If  such  were  the  case,  then  platform  motion  may 
have  a  greater  effect  for  narrow  field-of-vlew  visual  systems.  To  Investigate  this  hypothesis, 
wataupsky  et  al.  (45)  conducted  a  transfer  of  study  varying  motion  and  field-of-vlew.  As  noted  above, 
four  groups  received  four  ASPT  training  sorties  followed  by  a  data  ride  in  the  T-37  aircraft.  The 
aircraft  data  revealed  no  reliable  effects  due  to  motion,  field-of-vlew,  or  their  interaction.  Within 
the  simulator,  data  collected  on  each  of  the  four  sorties  revealed  no  reliable  effects  due  to 
field-of-vlew  or  its  interaction  with  motion.  The  motion  groups  performed  significantly  better  for  the 
Takeoff,  Slow  Flight  and  Straight-in  Approach/Landing  as  measured  by  IP  ratings  and  for  the  Straight-in 
Approach/Landing  as  scored  by  the  ASPT  automated  measurement  systems.  Since  these  efforts  were  evident 
on  the  first  measured  trial  and  no  slgnficant  motion  by  trial  interactions  were  found,  it  seems  likely 
there  existed  Initial  group  filfferences. 

Gray  and  Fuller  (60)  studied  the  contribution  of  platform  motion  to  the  acquisition  of  weapons 
delivery  skills  and  its  subsequent  transfer  to  the  aircraft.  As  previously  discussed,  the  training  was 
highly  successful.  With  regard  to  platform  motion,  no  differences  were  found  for  either  performance  in 
the  simulator  or  subsequent  performance  in  the  aircraft.  It  must  be  remembered,  however,  that  training 
was  provided  in  a  T-37  simulator  while  the  transfer  evaluation  was  conducted  in  the  F-5  aircraft. 
Although  it  Is  clear  that  the  addition  of  platform  motion  during  simulator  training  did  not  enhance  the 
transfer  to  the  aircraft,  the  generality  of  such  findings  is  questionable  due  to  the  dissimilarity  of 
aircraft  dynamics  during  training  and  evaluation. 

Pohlmann  and  Reed  (58)  attempted  to  determine  the  value  of  platform  motion  cueing  In  the  acquisition 
of  basic  air  combat  skills.  Oata  collected  during  aircraft  evaluations  revealed  the  training  to  be 
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ineffective.  In  fact,  the  trend  was  toward  better  performance  by  the  Control  Group  which  received  no 
simulator  pretraining.  Since  transfer  of  training  was  not  demonstrated,  data  bearing  on  the  motion 
issues  were  not  considered  meaningful. 

DISCUSSION 

Taken  at  face  value,  the  literature  suggests  that  the  addition  of  a  visual  system  will  enhance  the 
training  value  of  the  simulator  whereas  the  addition  of  a  platform  motion  system  will  have  little 
effect.  However,  there  are  dangers  in  attempting  to  draw  conclusions  from  diverse  and  often  unrelated 
research  studies.  The  research  goals  in  these  studies  vary  widely;  the  experimental  design  and 
measurements  are  different.  Each  of  these  factors  will  have  an  effect  {usually  unknown)  upon  the  study 
outcome.  In  the  following  section,  the  effect  of  study  design  factors  will  be  addressed. 

Research  Objectives 

At  the  outset,  the  stated  intent  was  to  review  the  literature  regarding  the  training  effectiveness 
of  motion  and  visual  simulation.  Training  effectiveness  was  defined  in  terms  of  enhanced  performance  in 
the  airborne  environment  as  a  result  of  simulation  training--in  other  words,  transfer  of  training.  It 
should  be  readily  apparent  that  only  some  of  the  reviewed  studies  have  attempted  to  directly  address 
this  question.  The  best  examples  have  been  efforts  addressing  the  contributions  of  platform  motion  to 
training  effectiveness  (56,  58,  60,  65).  In  those  studies,  training  was  given  under  alternative  motion 
cueing  conditions  and  compared  with  the  results  of  Identical  training  without  motion.  A  comparison  of 
performance  between  such  groups  enables  one  to  directly  assess  the  effect  of  the  motion  cueing.  Such  an 
approach,  however,  has  not  been  used  to  evaluate  the  effectiveness  of  visual  systems.  In  many 
instances,  it  would  not  be  warranted;  e.g.,  tasks  in  which  external  visual  cues  are  absolutely 
necessary,  such  as  formation,  aerial  refueling,  and  air  combat. 

However,  many  visual  tasks,  especially  for  transition  training,  have  or  can  have  a  large  instrument 
component.  Many  of  these  tasks  can  be  flown  from  cockpit  instruments  even  though  the  intent  is  to  make 
use  of  external  visual  cues.  In  the  event  that  the  visual  cues  are  not  adequate,  pilots  will  resort  to 
the  use  of  instruments.  It  seems  likely  that  because  of  this  large  instrument  component  of  transition 
tasks,  estimates  of  visual  training  effectiveness  may  be  inflated.  To  date,  the  authors  are  unaware  of 
any  transfer  studies  which  have  been  completed  wherein  one  control  group  received  training  for  the  same 
tasks  under  instrument  conditions  only.  Until  such  efforts  have  been  accomplished,  the  actual  benefits 
of  visual  simulation  training  for  transition  will  remain  unknown. 

Furthermore,  sane  of  the  studies  reviewed  were  concerned  with  the  evaluation  of  the  effectiveness  of 
simulation  training  in  which  the  visual  training  per  se  represented  only  a  fractional  part.  In  most 
instances,  evaluations  have  centered  around  a  single  system.  Only  a  few  efforts  have  attempted  to  study 
differential  transfer  as  a  function  of  visual  system  characteristics  (44,  45,  49),  despite  the  fact  that 
such  information  is  vital  to  the  procurement  process.  Consequently,  there  is  ample  evidence  that  visual 
simulation  training  is  effective,  but  little  data  to  guide  decisions  on  the  visual  system  requirements 
for  specific  applications. 

Experimental  Design  and  Control 

In  addition  to  differing  research  objectives,  there  are  also  differences  in  experimental  design 
which  characterize  the  literature.  Most  studies  have  been  designed  to  evaluate  the  effectiveness  of  a 
single  simulator  training  program.  In  such  situations,  the  desired  approach  has  been  to  train  to 
proficiency  in  both  the  simulator  and  the  aircraft.  Estimates  of  transfer  effectiveness  could  be 
obtained  by  comparison  with  a  group  trained  to  proficiency  in  the  aircraft  only.  Despite  the 
desirability  of  this  approach,  it  has  been  used  in  only  a  few  studies  (32,  36,  61).  Some  studies  have 
trained  the  experimental  group  to  proficiency  In  the  simulator  and  aircraft  and  subsequently  made 
comparisons  against  the  hours  in  the  normal  syllabus  (40,  42).  Other  studies  have  defined  a  fixed 
number  of  simulator  and  aircraft  hours  and  made  comparisons  against  a  "standard"  syllabus  in  terms  of 
final  proficiency  evaluations  (26,  33,  37,  47,  54).  Still  others  have  employed  a  fixed  number  of 
simulated  and  aircraft  hours  or  sorties  (34,  38,  39,  50,  52,  57). 

Studies  of  differential  transfer,  that  is,  comparing  different  simulator  training  conditions, 
present  an  added  problem.  The  investigator  has  the  option  of  either  training  the  simulator  groups  to 
proficiency  or  providing  a  fixed  number  of  trials.  While  training  to  proficiency  should  theoretically 
optimize  the  transfer,  it  makes  Interpretation  difficult  in  the  event  there  are  differences  in  trials  to 
criterion  in  the  simulator  as  well  as  differences  in  aircraft  performance.  In  such  instances,  the 
variable  of  Interest  is  confounded  with  training  time.  Furthermore,  there  is  the  added  danger  that 
training  both  groups  to  proficiency  in  the  simulator  may  enhance  the  likelihood  of  no  differential 
transfer.  On  the  other  hand,  use  of  a  fixed  trial  procedure  may  reduce  the  overall  effectiveness  of  the 
training,  thereby  increasing  the  variability  of  subsequent  aircraft  performance  and  reducing  the  power 
of  the  design.  Despite  this  danger,  most  studies  of  differential  transfer  have  used  a  fixed  training 
procedure  (43,  44,  45,  56,  58,  60).  Only  two  studies  (40,  42)  used  a  training  to  proficiency  approach, 
and  In  each  case,  the  differential  transfer  aspects  were  only  a  secondary  consideration.  Thorpe  et  al. 
(49)  used  a  combination  procedure  in  which  each  pilot  received  a  fixed  number  of  simulator  training 
sorties  unless  proficiency  criteria  were  reached  earlier. 

There  are  also  differences  in  terms  of  the  degree  of  experimental  control  exercised  during  simulator 
training.  Primarily,  two  approaches  have  been  used.  Some  studies  have  provided  simulator  Instruction 
in  a  manner  equivalent  to  operational  training.  No  special  procedure  or  sequence  of  training  was 
followed.  Other  studies,  however,  have  attempted  to  strictly  control  the  instructional  process  in  terms 
of  a  fixed  sequence  and  number  of  events  or  specific  criteria  for  advancement  to  the  next  task.  For  the 
most  part,  these  studies  concerned  with  differential  transfer  have  attempted  to  rigorously  control  the 
content  and  sequence  of  the  instructional  syllabus  whereas  those  evaluating  the  training  effectiveness 
of  a  single  systea  have  used  the  more  traditional  operational  approach.  However,  for  some  studies,  the 
report  did  not  provide  sufficient  Information  so  that  it  seems  likely  that  few  special  Instructional 
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control  procedures  were  followed.  The  extent  to  which  such  differences  affected  study  outcomes  is 
unknown. 

Proficiency  Assessment 

Perhaps  the  most  critical  aspect  of  the  transfer  of  training  study  is  the  assessment  of  aircrew 
performance.  The  use  of  reliable,  valid,  and  sensitive  indices  of  proficiency  is  essential. 
Measurements  are  needed  to  determine  proficiency  in  both  the  simulation  and  airborne  environments.  In 
the  studies  surveyed,  measurements  have  ranged  in  sensitivity  from  attrition  data  to  deviations  from  a 
desired  glidepath  as  measured  by  sophisticated  radar  equipment.  Despite  this  wide  range,  most  studies 
have  relied  upon  judgements  of  experienced  flight  instructors.  Some  studies  have  attempted  to  minimize 
the  subjective  aspects  of  instructor  evaluation  by  requiring  him  to  "record"  performance  rather  than 
“evaluate"  performance.  In  these  cases,  observations  such  as  maximum/minimum  altitude,  airspeed  at 
touchdown,  etc.,  would  be  recorded.  This  approach  was  used  in  some  of  the  earlier  University  of 
Illinois  studies  in  which  proficiency  was  defined  in  terms  of  these  behavioral  criteria.  Despite  the 
desirable  objectivity  of  such  an  approach,  it  requires  specialized  instructor  training  to  be  used 
successfully.  Furthermore,  the  possibility  remains  that  important  indicators  of  proficiency  may  be 
overlooked  using  this  approach. 

Other  studies  have  attenpted  to  capitalize  on  the  expertise  of  the  flight  instructor  and  incorporate 
his  judgment  into  the  flight  evaluation.  In  some  instances,  he  has  been  asked  to  evaluate  performance 
along  a  continuum  from  unsatisfactory  to  excellent  {26,  43,  45,  54,  58,  61).  Other  studies  have 
required  instructors  to  evaluate  performance  in  relation  to  some  normative  criteria,  e.g.,  the  top  3X  of 
students  you  have  instructed  (38,  49,  50,  57). 

Only  a  few  studies  have  made  use  of  automated  objective  scoring  procedures  wherein  no  instructor 
judgments  were  required.  Objective  in-simulator  performance  scoring  has  been  used  for  basic  contact  and 
approach/ landing  skills  (43,  44,  45)  and  weapons  delivery  training  (52,  60).  In  the  aircraft,  even 
fewer  studies  have  used  objective  data.  Brictson  and  Burger  (50)  recorded  glidepath  data  for  a  portion 
of  the  pilots  using  radar  equipment.  Objective  bomb  delivery  scores  were  used  by  the  two  previously 
mentioned  surface  attack  studies  (52,  60). 

Each  of  the  techniques  discussed  thus  far  has  been  applicable  to  the  evaluation  of  performance  on  a 
repetition-by-repetition  basis.  In  other  words,  a  student's  performance  might  be  considered  Good  on  the 
first  trial.  Fair  on  the  second.  Good  on  the  third,  and  so  on.  The  demonstration  of  proficiency  on  one 
trial  does  not  guarantee  the  same  level  of  performance  on  the  next.  The  definition  of  proficiency  in 
terms  of  continued  acceptable  performance  creates  additional  problems.  Some  studies  have  resorted  to  a 
single  instructor  judjnent  as  to  when  the  student  is  considered  “proficient"  (42,  47,  48,  61).  Other 
studies  have  defined  proficiency  in  terms  of  a  set  number  of  task  repetitions,  each  of  which  had  met 
proficiency  criteria.  For  example,  Payne  et  al.  (36)  required  three  successive  repetitions  in  which  all 
criteria  were  met  on  each  trial.  Thorpe  et  al.  (49)  required  five  successive  repetitions.  Other 
studies  using  a  fixed  number  of  training  trials  or  evaluation  sorties  have  not  had  to  develop  such  an 
overall  definition  of  proficiency. 

Sample  Size 

Reported  sanple  sizes  have  varied  substantial ly.  They  have  ranged  from  a  low  of  four  subjects  per 
group  (42)  to  a  high  of  100  subjects  per  group  (38).  The  choice  of  sample  size  is  usually  dictated  by 
economic  and  operational  constraints  rather  than  measurement  sensitivity  and  the  desired  power  of  the 
experimental  design.  The  relationship  between  behavioral  variability,  measurement  sensitivity  and  the 
sanple  size  is  straightforward.  Greater  variability  of  performance  and  reduced  measurement  sensitivity 
lead  to  a  requirement  of  larger  sample  sizes.  Failure  to  increase  the  sample  size  will  reduce  the  power 
of  the  test  to  detect  differences  in  the  event  they  actually  exist.  This  is  especially  critical  for 
relatively  small  effects.  It  should  be  apparent  that  studies  of  differential  transfer,  e.g.,  a 
comparison  of  alternate  visual  systems  or  motion  versus  no  motion  are  most  vulnerable  to  this  problem. 
The  effect  of  simulation  training  vs.  no  training  is  likely  to  be  substantially  larger  than  training  in 
System  A  vs.  training  in  System  8.  Therefore,  studies  of  differential  transfer  require  a  larger  number 
of  subjects  to  maintain  a  certain  degree  of  power  given  the  same  training  and  measurement  procedures. 

A  survey  of  the  reviewed  literature,  however,  reveals  just  the  opposite.  In  general,  studies  of 
differential  transfer  have  used  smaller  sample  sizes.  The  extent  to  which  these  sample  sizes  have  led 
to  the  predominant  finding  of  "no  differences"  is  unknown  since  such  efforts  have  generally  attempted  to 
exert  greater  experimental  control  thereby  reducing  behavioral  variability  and  increasing  statistical 
power.  Perhaps  It  is  safe  to  conclude  only  that  larger  saitple  sizes  would  have  been  desirable. 

Task  Selection 

For  most  transfer  of  training  evaluations  of  an  operational  simulator  training  system,  the  selection 
of  tasks  to  be  trained  does  not  present  a  major  problem.  In  most  Instances,  Instructors  fly  the 
simulator  to  subjectively  determine  which  tasks  can  be  realistically  flown.  Based  on  their  opinions,  a 
training  syllabus  Is  developed  and  subsequently  evaluated.  For  differential  transfer  studies,  however, 
the  selection  of  tasks  to  be  trained  presents  some  interesting  questions. 

The  strategy  of  most  differential  transfer  of  training  studies  has  been  to  select  tasks  which  are 
relevant  to  the  question  of  Interest,  provide  intensive  training  for  only  those  tasks,  and  evaluate  the 
transfer  to  the  aircraft.  In  coaparing  three  visual  systems  for  KC-135  training,  Thorpe  et  al.  (49) 
selected  the  circling  approach  and  landing  for  training.  Since  this  task  is  the  most  critical  and 
visually  dependent  task  flown  in  the  KC-135,  such  a  choice  seemed  appropriate.  Likewise,  Martin  and 
Cataneo  (44)  chose  takeoffs  and  landings  for  a  conparlson  of  Day  vs  Night  training  using  a  narrow 
fleld-of-vlew  visual  presentation.  Again,  such  a  choice  seems  reasonable  since  they  are  the  two  most 
Important  tasks  requiring  visual  cueing  which  are  trained  in  Air  Training  Conmand’s  new  Instrument 
Flight  Simulator. 
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Nataupsky  et  al.  (45),  in  an  effort  to  determine  the  interactive  effects  of  motion  and 
field-of-view,  chose  the  takeoff,  slow  flight,  steep  turn  and  straight-in  approach  and  landing  for 
training.  Since  the  primary  visual  cues  for  these  tasks  are  located  directly  in  front  of  the  simulated 
aircraft,  it  is  questionable  whether  they  were  good  choices  for  evaluating  field-of-view  effects. 
Likewise,  the  choice  of  tasks  to  evaluate  the  contributions  of  platform  motion  to  training  effectiveness 
has  stirred  controversy.  Two  types  of  motion  cueing  have  been  distinguished,  force  cues  resulting  from 
pilot  input  and  force  cues  resulting  from  environmental  or  aircraft  configuration  changes  (66).  The 
first  type  has  been  referred  to  as  maneuver  motion;  the  second,  disturbance  motion.  The  studies  to  date 
have  focused  primarily  on  tasks  having  a  large  maneuver  motion  cortponent.  Since  there  is  some 
in-simulator  performance  data  to  suggest  that  motion  may  not  enhance  the  performance  of  such  tasks  under 
stable  aircraft  conditions,  the  selection  of  such  tasks  to  evaluate  motion  cueing  has  been  questioned. 

General izability 

One  of  the  key  issues  in  any  research  effort  is  the  extent  to  which  the  results  have  application 
beyond  the  immediate  conditions  of  the  study.  This  requires  the  investigator  to  have  an  understanding 
of  the  critical  dimensions  which  may  impact  the  study  outcome  and  thereby  generate  a  design  which  will 
maximize  the  generality  of  the  results.  Although  it  is  known  that  factors  such  as  aircraft  type,  pilot 
experience  level,  and  type  of  task  may  affect  the  outcome,  little  attempt  has  been  made  to  integrate 
these  in  some  coherent  fashion.  Furthermore,  there  has  been  a  failure  to  quantify  the  critical 
dimensions  of  motion  and  visual  systems,  except  in  the  most  rudimentary  way  (e.g.,  On  vs.  Off  or  Day  vs. 
Night,  etc.).  In  other  words,  there  exist  no  quantifiable  models  of  visual  and  motion  simulation  which 
enable  testable  hypotheses  to  be  generated  which  subsequently  might  lead  to  general izable  findings. 
Until  such  models  are  developed,  progress  will  occur  in  a  precarious  fashion  at  best. 

A  look  at  the  studies  to  date  may  provide  some  understanding  of  the  failure  to  provide  a  set  of 
general izable  findings.  Most  transfer  of  training  evaluations  have  been  problem-oriented.  Studies  have 
been  done  to  answer  specific  questions.  What  is  the  value  of  a  night  carrier  landing  trainer?  Can 
simulator  time  be  substituted  for  P-3  aircraft  time  without  a  decrease  in  proficiency?  Which  is  the 
best  available  visual  system  for  the  KC-135  simulator?  Are  platform  motion  systems  required  for  fighter 
simulators?  The  research  conmunity,  in  its  attempt  to  provide  "real-world"  solutions  for  today's 
problems,  has  failed  to  develop  the  framework  for  obtaining  data  for  tomorrow's  issues. 

The  Fidelity  Issue 

A  key  exanple  of  a  failure  to  "look-ahead"  concerns  the  fidelity  of  simulation  necessary  to  insure 
transfer  of  training.  There  is  no  question  that  for  many  tasks  full  fidelity  simulation  is  not 
necessary.  Pilots  have  been  aided  by  very  low  fidelity  trainers  for  years.  Furthermore,  many  of  the 
research  studies  cited  in  this  report  clearly  document  the  fact  that  the  flight  simulator  does  not  have 
to  duplicate  the  aircraft  in  order  that  training  be  effective.  If  full  fidelity  is  not  necessary,  then 
exactly  how  much  is  required?  Unfortunately,  this  question  cannot  be  answered  until  other  issues  are 
addressed. 

First,  at  a  very  basic  level,  what  skills  transfer  from  the  simulator  to  the  aircraft?  It  is 
observed  that  the  transfer  for  some  tasks  is  quite  high;  for  others,  quite  low.  Little  is  known 
regarding  the  underlying  basis  of  these  observed  differences.  Basic  research  is  needed  which  clearly 
identifies  those  elements  of  simulator  training  which  transfer  to  the  aircraft.  For  example, 
conputer-generated  imagery  visual  systems  are  often  very  cartoonish.  Yet,  there  is  evidence  to  suggest 
that  they  provide  more  effective  training  than  terrain  model  board  systems  which  more  closely  duplicate 
the  real  world  environment.  It  is  apparent  that  the  key  variable  is  not  the  physical  fidelity  of  the 
system.  Research  is  needed  to  identify  those  critical  elements  which  do  account  for  these  observed 
transfer  effects. 

Once  these  critical  transfer  elements  have  been  defined,  it  is  necessary  to  derive  the  relationship 
between  the  degree  of  fidelity  and  the  amount  of  transfer.  It  is  at  this  point  that  trade-offs  can  be 
generated  between  costs  associated  with  increased  fidelity  and  costs  associated  with  providing  training 
in  the  aircraft.  It  may  be  that  for  some  tasks,  the  aircraft  is  the  most  cost  effective  training 
device.  Until  this  Information  is  available  and  a  valid  cost-effectiveness  model  developed,  the  queston 
of  how  much  fidelity  is  needed  will  remain  unanswered.  Because  of  our  current  Inability  to  match 
training  and  fidelity  requirements,  it  is  likely  that  simulators  will  continue  to  be  procured  under  the 
design  goal  of  maximum  fidelity. 


Vi.  CURRENT  PRACTICES  IN  THE  USE  OF  SIMULATORS 


DEVELOPMENT  OF  SIMULATOR  TRAINING  PROGRAMS 

Development  of  plans  for  use  of  a  simulator  In  a  flying  training  program  typically  precedes  the 
purchase  of  flight  simulation  devices  which  provide  capabilities  for  training,  practice  and  evaluation 
of  flying  tasks  which  could  not  be  trained  using  existing  simulation  devices  of  the  to-be-revised 
training  program.  Under  these  circumstances,  a  fresh  look  at  the  total  training  program  package  often 
follows  and  some  form  of  a  training  system  analysis  may  be  made  If  enough  time  and  money  are  available. 
The  objective  of  such  an  analysis  Is  to  Increase  the  efficiency,  economy,  safety  or  other  salient 
attributes  of  the  overall  training  operation.  The  analysis  may  reveal  opportunities  for  program  changes 
which  will  achieve  various  training  Improvements.  Since  Inclusion  of  a  new  simulator  guarantees  a 
revision  of  an  existing  training  syllabus,  other  productive  modifications  to  training  often  can  be 
included  at  little  extra  cost  in  time  or  money. 


SIMULATOR  AS  AIRCRAFT  SUBSTITUTE 


At  this  point,  the  requirements,  intentions,  and  concepts  of  the  training  managers  very  likely 
determine  how  the  simulator(s)  will  be  employed  within  the  new  training  program.  8y  considering  the 
potential  of  the  simulators  in  question,  the  training  manager  may  conceive  of  them  as  taking  the  place 
of  the  aircraft.  This  approach  asks  the  question,  "What  tasks  can  be  trained  in  the  simulator?"  Once 
the  tasks  that  can  be  trained  in  the  simulator  are  identified,  the  program  is  modified  accordingly, 
leaving  the  remaining  training  tasks  to  be  taught  using  the  aircraft.  This  type  of  application  of 
simulators  is  simple,  straightforward,  practical,  effective  and  often  results  in  gratifying  energy  and 
cost  savings  through  reductions  in  flying  hours  required  for  training.  Diehl  and  Ryan  (67)  have 
produced  an  extensive  review  of  research  results  to  demonstrate  the  effectiveness  of  the  simulator  as 
substitute  for  the  aircraft  in  flying  training. 

An  exaiqple  of  this  kind  of  non- integrated  training  program  development  can  be  seen  in  the  US  Air 
Force  Air  Training  Command's  (ATC)  new  Instrument  Flight  Simulator  (IFS).  The  IFS  will  be  used  to 
provide  instrument  procedures  training  in  both  T-37  and  T-38  phases  of  UPT.  Training  in  the  IFS  will 
replace  aircraft  instrument  training  with  the  exception  of  the  instrument  phase  checks  which  will  be 
given  in  the  appropriate  aircraft.  The  ATC  syllabus  (68)  shows  a  reduction  of  about  35  flying  hours  as 
a  result  of  use  of  the  IFS  for  all  simulator  instrument  training. 

INTEGRATED  SIMULATOR  TRAINING 

In  contrast,  a  training  manager  with  an  analysis  of  the  overall  training  situation  before  him  may 
conceive  of  flight  simulators  as  additional  training  aids  rather  than  as  substitutes  for  the  aircraft. 
In  this  case,  he  may  consider  a  variety  of  training  media  which  vary  in  cost,  complexity  and 
representation  of  the  in-flight  environment  and  evaluate  a  variety  of  ways  to  arrange  their  use  to 
generate  an  effective  training  program.  In  this  type  of  program,  new  training  material  is  introduced 
through  lecture/discussion,  some  form  of  group  instruction,  or  through  the  use  of  media-oriented 
individualized  instruction.  This  is  followed  by  a  progress  check  and  additional  instruction  as 
appropriate  to  achieve  the  training  objectives.  Part-task  training  aids  then  may  be  employed  to  provide 
realistic  practice  in  limited  operation  of  equipment  systems.  In  successive  lessons,  the  learner 
acquires  the  ability  to  perform  the  tasks  required  of  him  one  by  one,  and  as  he  learns,  he  can  be 
instructed  in  the  way  these  part-tasks  are  integrated  in  real  time  in  accomplishing  the  criterion  task 
he  is  learning  to  perform. 

Once  the  part-tasks  are  mastered,  the  learner  is  prepared  to  take  maximum  advantage  of  practice 
opportunities  in  the  flight  simulator,  which  he  is  now  ready  to  deal  with  competently  and  productively. 
In  the  simulator  the  learner  can  practice  the  entire  task  in  real  time,  evaluate  his  performance,  make 
corrections  in  the  way  he  handles  the  demands  of  the  task  and  eventually  acquire  the  skills  needed  to 
perform  at  the  required  level  of  proficiency.  At  this  point,  the  trainee  may  be  evaluated  in  the 
simulator  and  subsequently  in  the  aircraft  or  given  his  checkride  directly  in  the  aircraft.  Thus  the 
check/evaluation  routine  becomes,  in  addition,  a  validation/verification  of  training  success.  Following 
completion  of  check  flight,  the  learner  may  proceed  to  the  next  phase  which  can  be  organized  in  much  the 
same  way.  Should  the  trainee  not  pass  his  demonstration/check  exercise,  he  and  his  instructor  can 
determine  what  additional  practice  is  needed  and  return  to  the  simulator  to  refine  the  performance  until 
he  is  ready  for  reevaluation.  Such  a  sequence  illustrates  one  of  the  simulator's  chief  instructional 
virtues,  its  capability  to  create  specific  training  exercises  which  can  be  tailored  to  match  a  trainee's 
specific  need  for  practice. 

An  exanple  of  the  integrated  use  of  simulators  in  flying  training  can  be  seen  in  the  American 
Airlines'  Transition  and  Recurrency  programs  for  Captains  and  First  Officers  (46,  69,  70).  The  American 
Airlines  program  begins  with  study  of  technical  information  using  handbooks,  manuals  and  a  variety  of 
audiovisual  media.  System  operations  training  is  then  given  using  part-task  trainers  in  addition  to 
verbal  and  visual  instruction.  Trainees  progress  through  the  program  on  a  proficiency  basis.  They 
complete  a  sequence  of  training/practice  flights  in  an  appropriate  simulator  where  they  demonstrate 
their  readiness  to  pass  an  American  Airlines  evaluation  in  the  aircraft.  Following  this  evaluation,  the 
trainee  routinely  is  given  an  FAA  certification  check. 

RECONSIDERING  THE  ALTERNATIVES 

Both  substitution  and  integrated  simulator  training  have  substantial  systematic  strength  and  both 
have  demonstrated  their  utility  and  effectiveness  (46,  67,  70).  The  simulator  substitution  approach 
tends  to  emphasize  the  intuitively  obvious  fact  that  the  simulator  is  not  really  like  the  aircraft  and 
therefore  may  be  viewed  as  a  less  than  desirable  substitute.  In  such  cases,  the  result  may  be  less  than 
optimum  utilization  of  the  simulator  due  to  the  its  relative  lack  of  realism.  Thus  the  non- integrated 
approach  tends  to  limit  the  effectiveness  of  a  simulator's  application  as  a  teaching  tool  because  of  its 
non- identity  with  the  aircraft. 

On  the  other  hand,  the  integrated  approach  to  simulator  utilization  in  flying  training  defines  a 
teaching  role  for  the  simulator  which  is  based  on  its  instructional  strengths.  In  this  instance,  the 
simulator's  training  effectiveness  does  not  depend  on  its  aircraft-like  flying  characteristics, 
fidelity/realism  or  other  similar  attributes  which  in  nearly  every  case  will  drive  the  cost  and 
complexity  of  a  simulator  higher  than  is  needed  In  order  for  it  to  be  used  effectively  for  transfer  of 
training  in  the  aircraft  (3).  Consequently,  the  Integrated  simulator-aircraft  flying  training  program 
is  less  vulnerable  to  unwarranted  criticism,  may  generate  greater  instructor  and  trainee  acceptance  and 
lead  to  higher  cost-effectiveness  than  non- integrated  programs.  To  conclude  this  discussion  of  current 
alternative  simulator  applications,  It  should  be  noted  that  regardless  of  the  orientation  of  ihe 
training  managers  and  instructional  system  developers.  Instructors  typically  use  the  simui^cors 
effectively  and  will  learn  to  "train-around"  their  limitations  to  exploit  their  capabilities  and 
minimize  their  deficiencies  (71). 
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Vll.  FLIGHT  SIMULATORS  USED  FOR  EMERGENCY  PROCEDURES  TRAINING 


CURRENT  EMERGENCY  PROCEDURES  TRAINING 

With  many  of  today's  aircraft  systems,  particularly  transport  and  fighter-type  aircraft,  there  are  a 
number  of  emergency  conditions  that  can  arise  during  flight  operations  which  are  too  dangerous  to  train 
in  the  aircraft.  Engine  failure  on  takeoff  or  landing  or  various  maneuvers  associated  with  departure 
from  controlled  flight  are  examples  of  such  emergency  conditions.  From  the  beginning,  one  of  the 
signficant  strengths  of  the  flight  simulator  has  been  to  support  training  in  procedures  for  dealing  with 
emergencies  which  cannot  be  learned,  practiced,  or  evaluated  safely  in  the  aircraft.  As  a  consequence, 
the  use  of  flight  simulators  for  emergency  procedures  training  has  increased  steadily  and  development  of 
simulator  technology  has  included  attention  given  to  increasing  the  scope  and  quality  of  emergency 
procedures  training  capabilities  in  state  of  the  art  devices. 

The  emergency  conditions  which  can  occur  in  an  aircraft  system  and  the  aircrew  procedures  specified 
to  deal  with  them  properly  are  included  routinely  with  presentation  of  technical  information  on  normal 
operation  conditions  and  procedures.  Often  the  determination  that  an  emergency  condition  exists  is 
based  on  detection  of  an  operational  parameter  above  or  below  the  normal  range;  for  example,  high 
exhaust  gas  temperature  or  low  oil  pressure.  Other  types  of  emergencies  are  signaled  by  a  discontinuity 
in  a  normal  operation,  such  as  engine  failure  or  electrical  system  malfunction. 

THE  SEQUENCE  OF  EMERGENCY  PROCEDURES 

In  any  case,  an  emergency  condition  is  defined  in  terms  of  the  information  available  to  the  crew  for 
use  in  determining  when  an  emergency  exists.  The  emergency  procedures  which  are  required  to  correct  the 

emergency  conditions  are  defined  as  a  sequence  of  actions  to  be  taken  by  the  crew  which  include  the 

detection  and  recognition  of  additional  signals,  to  be  checked  by  the  crewmember  to  permit  him  to 
evaluate  the  effects  of  the  emergency  procedures '"he  has  accomplished.  The  desired  sequence  of  aircrew 
actions  in  the  event  of  an  aircraft  emergency  condition  are: 

1.  Detect  the  indication  that  an  emergency  condition  exists. 

2.  Recognize  the  specific  characteristics  of  the  emergency. 

3.  Recall  the  specific  procedures  to  be  accomplished  to  correct  the  emergency  condition. 

4.  Accomplish  the  emergency  procedures  promptly  and  accurately. 

5.  Make  an  appropriate  assessment  of  the  effects  of  the  result  of  the  corrective  actions  and 

accomplish  additional  emergency  procedures  if  indicated  (72). 

In  practice,  emergency  procedures  training  consists  of  memorizing  specific  sequences  of  actions. 

Mastery  of  emergency  procedures  is  verified  in  a  verbal  or  written  classroom  test  or  in  a  simulator  test 
exercise. 

In  classroom  and  simulator  emergency  procedures  tests,  practice  may  occur  only  if  the  trainee  fails 
to  exhibit  the  exact  correct  verbal  responses  or  procedural  actions  required.  This  is  a  result  of  the 
relatively  simple  characteristics  of  many  emergency  procedures  which  require  substantially  more  rote 
memory  than  perceptual-motor,  discrimination  or  cognitive  skills.  Since  some  emergency  conditions  may 
be  corrected  more  easily  if  they  are  detected  quickly,  speed  in  completing  an  emergency  procedure  is 
rated  highly  provided  no  procedural  errors  are  made.  Thus,  it  can  be  seen  that  emergency  procedure 
training  involves  learning  the  sequence  of  action  requirements  and  practicing  the  procedures  until  they 
can  be  performed  quickly  and  without  error  following  the  first  signal  of  an  emergency  condition. 

SIMULATOR  EMERGENCY  TRAINING 

In  the  simulator,  emergency  procedure  training  Involves  periodic  testing  of  the  single  procedural 
sequences  for  speed  and  accuracy.  Depending  on  the  characteristics  of  the  simulator  in  which  the 
emergency  procedures  are  practiced  and  evaluated,  the  signal  of  an  emergency  condition  may  be  programmed 
as  a  normal  element  in  a  broader  simulator  training  session.  For  example,  in  a  session  which  Involves 
instriment  approach  and  let-down  procedures,  the  Instructor  may  decide  to  Insert  an  ILS  Indicator 
failure  between  the  inner  marker  and  breakout  under  the  overcast.  When  the  indicator  failure  occurs, 
the  trainee  will  be  evaluated  in  terms  of  his  proapt  detection  of  the  emergency  condition  and  his 
performance  of  such  aircraft  control  and  subsystems  management  tasks  as  are  specified  for  the  emergency 
that  has  been  simulated. 

At  this  point  in  the  description  of  the  use  of  simulators  for  emergency  procedure  training,  it  is 
appropriate  to  deal  with  reallsm/f idelity  issue  once  again  as  it  relates  to  generating  In  a  trainee  the 
ability  to  perform  an  emergency  procedure  in  the  context  of  a  substantial  mission  segment  under  the 
relatively  stressful  conditions  of  high  task  load.  In  emergency  procedure  training,  the  actual  realism/ 
fidelity  of  a  flight  simulator  is  unchanged.  The  situation  In  which  the  emergency  Is  presented, 

however,  may  be  made  more  realistic  and  the  resulting  situational  realism  may  be  the  key  to  effective 
simulator  emergency  procedure  training.  In  this  case,  situational  realism  does  not  depend  on  the 

engineering  accuracy  or  other  design  characteristics  of  the  simulator  but  Instead  on  the  reasonableness 
and  beltevablllty,  that  is,  the  cognitive  realism  of  the  emergency  condition  given  the  context  in  which 
It  is  presented  to  the  trainee. 

SITUATIONAL  EMERGENCY  TRAINING 

Having  thus  identlf '  d  the  cognitive  aspects  of  emergency  procedure  training.  It  should  be 

emphasized  that  the  simulator's  function  In  generating  cognitive  realism  Is  substantial  but  not 


:i 


critical.  That  this  is  in  fact  the  case  has  been  demonstrated  in  a  new  approach  to  emergency  procedures 
training  called  Situational  Emergency  Training  (SET)  (72).  This  approach  is  based  on  the  essential 
situational  aspects  of  the  emergency  condition  and  involves  only  a  simple  three-step  procedure  which  the 
pilot  must  accomplish  as  soon  as  an  emergency  condition  is  detected: 

1.  Maintain  control  of  the  aircraft. 

2.  Analyze  the  situation  and  take  the  most  appropriate  corrective  action. 

3.  Land  the  aircraft  as  soon  as  possible. 

SET  emphasizes  detection  of  an  emergency  condition  in  terms  of  normal  mission  activities  where  the 
pilot  either  suddenly  or  gradually  recognizes  the  presence  of  an  abnormal  state.  At  this  point,  he  main¬ 
tains  control  of  the  aircraft  and  analyzes  the  situation  which  may  be  relatively  simple  or  may  involve  a 
multi-step  information  seeking  and  signal  verification  process.  Once  the  trouble  is  clearly  understood, 
the  pilot  must  determine  the  appropriate  course  of  action  and  execute  it.  This  typically  involves 
anticipation  of  the  interaction  of  his  corrective  actions  with  the  solution  of  the  imnediate  problem  and 
the  end  goal  of  a  safe  landing.  Such  activities  represent  instances  of  the  effective  exercise  of 
judgment  under  stressful,  often  life  and  death  conditions.  It  appears  also  to  be  what  experienced 
pilots  do  under  actual  emergency  conditions.  It  is  clear  that  SET  is  focused  on  training  good  judgnent. 

Considering  the  properties  of  a  SET  program,  it  can  be  seen  that  the  emergency  procedure  training 
can  be  conducted  in  many  instructional  environments,  including  a  flight  simulator.  There  are 
indications  that  the  SET  program  has  been  effective  in  the  case  for  which  it  was  developed  (72).  Since 
emergency  procedure  training  has  been  successfully  conducted  in  simulators  using  the  older, 
procedures-oriented  approach,  there  is  no  reason  to  expect  that  the  SET  approach  will  not  work  as  well 
by  using  flight  simulators  to  more  fully  prepare  the  trainee  to  acquire  the  judgment  ability  needed  to 
deal  efficiently  and  safely  with  the  emergency  procedures  which  he  may  at  any  time  be  required  to 
accomplish  as  part  of  daily  flight  operations. 


V] II. FLIGHT  SIMULATOR  AIRCREW  PERFORMANCE  MEASUREMENT 


REQUIREMENTS  FOR  SIMULATOR  PERFORMANCE  MEASUREMENT 

Effective  utilization  of  aircrew  training  devices  is  dependent  upon  the  development  of  adequate 
measures  of  pilot  performance.  A  training  device  is  effective  to  the  extent  that  it  enhances  the 
pilot's  ability  to  fly  the  aircraft.  The  evaluation  of  transfer  of  training  requires  the  measurement  of 
his  proficiency,  in  both  simulator  and  airplane.  This  chapter  will  discuss  the  development  of  automated 
performance  measurement  (APM)  systems  for  application  to  flight  simulation.  Following  discussion  of  the 
requirements  for  APM  systems,  specific  application  to  two  devices— a  research  simulator  in  which  the  APM 
focuses  on  the  pilot  and  an  operational  full  mission  simulator  where  crew  positions  in  addition  to  the 
pilot  (and  co-pilot)  are  included  in  the  APM  system— will  be  presented. 

Performance  measurement  may  be  defined  as  a  set  of  rules  which  categorize  and/or  quantify  behavior 
in  terms  of  some  pre-defined  characteristic  or  attribute.  It  follows  then  that  a  clear  understanding 
and  precise  definition  of  what  is  to  be  measured  is  essential.  Thus  a  set  of  rules  which  assign  a 
specific  category  or  number  to  a  specified  instance  of  behavior  must  be  developed  and  careful 
consideration  must  be  given  to  the  definition  of  the  rules  so  that  the  measurement  process  becomes  as 
precise  as  possible.  These  characteristics  represent  the  what  and  how  of  measurement.  However,  the 
measurement  process  always  occurs  within  the  context  of  a  specific  environment  and  the  results  are  used 
by  specific  Individuals.  These  practical  considerations  represent  the  why  and  for  whom  of  measurement 
and  are  of  great  importance  since  they  will,  to  a  large  extent,  Influence  and  shape  the  what  and  how  of 
measurement.  The  design,  development,  and  implementation  of  APM  systems  for  flight  simulators  requires 
that  the  following  factors  be  considered:  (1)  the  definition  of  measurement  requirements;  (2)  the 
selection  of  tasks  for  which  measures  are  to  be  developed;  (3)  the  definition  of  performance  measures; 
(4)  the  development  of  task  segmentation  logic;  (5)  the  development  of  summary  assessment  scores;  (6) 
the  definition  of  performance  feedback  and  display  capabilities;  (7)  the  definition  of  data  storage  and 
analysis  capabilities;  and  (8)  system  configuration  and  implementation  considerations.  Each  of  these 
will  be  discussed  below. 

SIMULATOR  PERFORMANCE  MEASUREMENT  DEVELOPMENT 

Why  measure  aircrew  performance?  Considering  the  spectrum  of  military  aviation,  these  reasons  for 
aircrew  performance  measurement  are  suggested:  (1)  assessment  of  aircrew  proficiency;  (2)  training 
system  development;  (3)  training  device  evaluation/simulator  certification;  (4)  prediction  of  future 
aircrew  mission  effectiveness;  and  (5)  research  and  development.  Of  these,  the  assessment  of  aircrew 
proficiency  is  probably  the  most  Important,  especially  for  application  to  operational  training  devices. 
Measurements  of  flying  proficiency  reflect  the  degree  to  which  training  objectives  are  met  and  generally 
specifies  any  performance  errors  committed.  The  necessity  of  Insuring  that  all  aircrews  meet  minimum 
proficiency  standards  cannot  be  overemphasized.  The  ability  to  assess  proficiency  also  Is  necessary  for 
the  proper  design  of  training  programs. 

Since  most  flying  training  programs  are  a  mixture  of  ground-based  and  airborne  Instruction,  the 
assessment  of  proficiency  In  both  domains  is  necessary  for  the  development  of  a  training  system.  Thus 
measurement  of  aircrew  proficiency  Is  necessary  to  evaluate  the  effectiveness  of  training  devices  and 
the  program  within  which  they  are  used.  The  purpose  of  ground-based  training  Is  to  enhance  performance 
In  the  aircraft.  The  application  of  the  transfer  of  training  methodology  as  a  means  of  evaluating  the 


effectiveness  of  ground-based  training  is  dependent  upon  aircrew  proficiency  assessment.  It  follows 
tfien  that  certification  of  flight  simulators  in  terms  of  their  training  effectiveness  is  also  dependent 
upon  an  adequate  measurement  capability. 

A  closely  related  consideration  is  the  user  of  measurement  information.  Who  has  need  for  measures 
of  aircrew  performance?  An  inspection  of  flying  training  programs  suggests  that  a  variety  of  people  use 
such  information,  but  that  the  depth  or  content  may  vary  significantly.  The  basic  unit  of  flying 
training  remains  the  student  and  his  instructor.  The  greatest  need  for  accurate,  detailed  information 
is  at  this  level.  The  instructor  needs  diagnostic  information  which  he  can  use  to  evaluate  his 
student's  performance,  isolate  his  errors,  and  provide  effective  remediation.  In  addition,  the  student 
requires  the  same  diagnostic  information.  Furthermore,  he  needs  such  feedback  in  a  timely  manner  if  it 
is  to  be  maximally  effective. 

The  job  of  the  syllabus  designer  is  to  structure  an  optimum  training  syllabus.  Within  limitations, 
he  must  specify  which  training  tasks  are  to  be  performed,  the  sequence  of  these  tasks,  and  the  number  of 
repetitions  for  each.  Although  performance  measurement  information  is  necessary,  the  level  of  detail  is 
not  as  great  as  that  required  by  the  student  or  instructor.  In  most  instances,  the  syllabus  designer  is 
interested  in  the  proficiency  levels  demonstrated  for  each  task  using  the  existing  syllabus.  Such  data 
is  useful  for  reconmending  syllabus  revisions.  The  training  manager  also  uses  performance  measurement 
information  as  a  tool  for  making  administrative  decisions  which  could  impact  the  training  program.  In 
most  instances,  he  does  not  need  the  level  of  detail  required  by  the  instructor  or  syllabus  designer. 

In  addition  to  requirements  resulting  from  the  intended  use  and  user  of  an  APM  system,  there  are 
basic  measurement  requirements  which  must  be  met.  The  first  of  these  is  reliability.  Considering  the 
measurement  of  pilot  performance,  two  sources  of  variability  must  be  distinguished-variability  due  to 
the  measurement  process  and  variability  due  to  the  pilot.  Of  concern  here  is  variability  due  to  the 
measurement  process.  To  the  extent  that  application  of  the  measurement  process  to  the  same  behavior 
produces  the  same  number  or  category,  the  process  is  said  to  be  reliable. 

The  second  basic  requirement  is  validity.  Although  several  types  have  been  enumerated,  the  one  most 
appropriate  for  performance  measurement  is  content  validity.  It  demands  that  the  most  salient 
behavioral  components  be  incorporated  into  the  measure.  To  the  extent  that  the  measurement  addresses 
all  of  the  criterion-referenced  objective  descriptions  of  the  behavior  students  must  acquire  during 
training,  the  content  validity  will  be  high.  A  second  type  of  validity  is  predictive  validity.  A 
measurement  is  valid  to  the  extent  that  it  is  useful  in  predicting  some  outcome— for  example, 
proficiency  at  the  end  of  training,  combat  effectiveness,  etc.  For  tasks  in  which  the 
criterion-referenced  objectives  are  vague  or  unknown,  this  type  of  validity  assumes  greater  importance. 

A  third  basic  requirement  is  sensitivity.  The  resulting  measurement  must  be  sufficiently  sensitive 
for  its  intended  use.  Although  a  pass-fail  measure  may  be  of  sufficient  sensitivity  for  the  needs  of 
the  flying  training  manager,  it  would  be  of  little  value  for  the  instructor  or  syllabus  designer.  In 
this  case,  there  is  a  need  to  match  the  sensitivity  of  the  measurements  with  the  needs  of  the  user. 

The  selection  of  tasks  for  which  measures  will  be  developed  is  largely  a  function  of  the  intended 
use  and  user  of  the  APM  system.  At  the  most  elemental  level,  pilot  behavior  can  be  described  as  an 
integrated  sequence  of  continuous  display  scanning  and  control  movement  responses.  At  the  other 
extreme,  pilot  behavior  might  be  described  in  terms  of  mission  effectiveness  within  a  combat 
environment.  At  intermediate  levels,  the  individua’  task  such  as  a  barrel  roll  attack,  or  a  class  of 
tasks  such  as  air-to-surface  weapons  delivery  might  be  selected.  It  should  be  apparent  that  there  exists 
a  hierarchy  of  measurement.  As  one  moves  up  this  hierarchy,  measurements  tend  to  become  less  specific 
and  more  abstract  since  they  are  based  upon  more  information. 

The  definition  of  performance  measures  also  is  largely  a  function  of  the  intended  use  and  user  of 
the  APM  system.  Although  a  number  of  approaches  to  proficiency  assessment  have  been  proposed,  a 
criterion-referenced  methodology  appears  to  be  the  most  appropriate  within  the  flying  training 
environment.  The  criterion-referenced  approach  to  measurement  system  development  seeks  to  define  the 
behavioral  objectives  for  each  task  and  subsequently  to  measure  the  degree  to  which  these  objectives  are 
met.  Despite  the  success  of  this  approach,  for  many  aircrew  tasks,  the  criterion-referenced  objectives 
can  only  be  vaguely  defined,  so  that  the  resulting  measurements  are  not  sufficiently  sensitive.  In  these 
Instances,  research  is  required  for  the  development  of  appropriate  measurements. 

The  general  approach  taken  for  the  development  of  objective  performance  measures  has  been  to  analyze 
each  task  into  its  component  parts  and  define  objectives  for  each  component.  A  benefit  of  this  approach 
with  respect  to  measurement  development  is  that  it  is  usually  easier  to  define  behavioral  objectives  at 
such  a  level.  Although  a  mission  profile  or  an  individual  maneuver  is  a  continuous  event.  It  is  easier 
to  conceptualize  them  as  a  sequence  of  individually  defined  tasks  or  sub-tasks.  In  the  development  of 
rules  for  specifying  the  start  and  end  conditions  for  each  task,  it  is  critical  that  logic  rules 
unequivocally  define  when  a  task  has  been  initiated  and  when  it  has  been  conpleted.  The  development  of 
such  rules  which  can  be  translated  Into  computer  logic  presents  certain  problems.  Some  of  these  will  be 
addressed  in  a  subsequent  description  of  APM  system  development  efforts. 

The  analytical  breakdown  of  tasks  Into  their  coityanent  parts  requires  that  the  individual  measures 
of  performance  be  sumnarlzed.  Inherent  in  the  definition  of  an  overall  assessment  score  is  a  value 
judgment  concerning  the  goodness  or  badness  of  a  particular  performance.  In  other  words,  for  each 
measure  there  is  some  performance  standard  which  defines  an  "acceptable"  level  of  performance.  Since 
there  Is  usually  some  deviation  about  the  desired  values,  the  question  becomes  how  much  deviation  is 
"acceptable."  Unfortunately,  there  are  no  completely  acceptable,  universal  techniques  for  establishing 
standards  of  "acceptability." 


The  combination  of  the  Individual  measures  Into  a  meaningful,  single  score  also  presents  problems. 
For  example,  a  single  score  will  not  provide  information  as  to  which  parameter  is  producing  the  greatest 
deviation  from  an  ideal  flight  path.  Although  each  observed  flight  path  will  uniquely  define  a 
performance  score,  each  performance  score  does  not  uniquely  define  a  particular  flight  path.  A  given 
score  could  be  obtained  from  an  infinite  nunber  of  flight  paths.  Thus,  the  obtained  measurement 
provides  little  diagnostic  information,  and  since  there  are  no  research  data  which  can  guide  the  APM 
system  developer  in  properly  weighing  each  measure  to  arrive  at  a  summary  score,  the  problems  of 
developing  a  single  score  remains  unsolved. 

The  utility  of  information  generated  by  an  APM  system  depends  upon  how  well  it  is  displayed  to  the 
user.  The  "acceptance”  of  the  interface  between  the  system  and  the  user  will  greatly  impact  its 
perceived  utility.  Performance  feedback  requirements  are  greatest  whenever  the  APM  system  is  to  become 
an  integral  part  of  the  simulator  training  curriculum. 

Consideration  should  be  given  to  potential  data  storage  and  analysis  requirements.  The  amount  and 
types  of  information  to  be  saved  are  largely  dependent  upon  its  intended  use.  Trade-offs  are  required 
since  it  is  not  feasible  to  store  all  information  obtained  during  a  given  mission.  Consideration  also 
should  be  given  to  specific  identifiers  which  would  be  of  importance  for  subsequent  analyses.  Retrieval 
software  should  be  sufficiently  flexible  to  enable  the  user  to  analyze  only  the  data  he  desires.  For 
most  operational  applications,  simple  descriptive  analyses  should  be  sufficient. 

Aircrew  performance  in  the  simulator  can  be  scored  in  two  modes,  off-line  and  real-time.  In  the 
off-line  mode,  the  parameters  to  be  scored  are  recorded  onto  a  storage  mediun  such  as  magnetic  tape  or 
disc.  This  raw  data  then  may  be  processed  using  an  off-line  computer  program.  Although  this  approach 
is  of  value  for  initial  development  and  evaluation  of  scoring  algorithms,  it  is  of  little  value  for 
training  due  to  feedback  delays.  Real-time  measurement,  on  the  other  hand,  offers  immediate  feedback  to 
the  aircrew  and  eliminates  the  need  for  off-line  processing.  The  requirements  for  its  implementation, 
however,  are  much  greater.  First,  it  is  necessary  that  the  software  for  the  scoring  algorithms  to 
reside  on-line  with  the  basic  simulation  program.  To  do  this,  sufficient  memory  core  must  be  available 
for  the  added  software  as  well  as  sufficient  spare  time  so  that  the  software  can  be  raised  without 
interfering  with  the  basic  simulation  program.  In  addition,  it  is  necessary  that  the  sufficient 
peripherals  exist  so  the  results  can  be  displayed  and  stored. 

MEASUREMENT  SYSTEM  DEVELOPMENT  FOR  A  RESEARCH  SIMULATOR 

The  ASPT  was  developed  to  be  a  research  tool  capable  of  providing  answers  to  questions  concerning 
the  design  and  effective  utilization  of  advanced  flight  simulators.  Despite  the  sophistication  of  the 
ASPT  and  its  research  potential,  one  key  ingredient  was  lacking— an  objective  pilot  performance 
measurement  system.  The  development  and  implementation  of  an  APM  system  was  one  of  the  initial  efforts 
undertaken  following  delivery  of  the  ASPT  since  this  capability  would  become  the  foundation  of  future 
research  to  be  accomplished  in  the  device.  This  effort  is  described  in  Fuller,  Haag,  and  Martin  (73). 

As  noted  above,  the  ASPT  was  desighed  specifically  to  study  how  training  is  impacted  by  various 
simulator  configurations  and  techniques  and  how  such  training  transfers  to  the  aircraft.  Consequently, 
its  measurement  system  should  emphasize  the  most  salient  characteristics  of  the  training  process.  Two 
key  elements  in  the  development  of  instructional  systems  are  the  definition  of  criterion-referenced 
objectives  and  the  specification  of  performance  standards.  Thus,  the  measurement  system  must  provide 
information  on  the  degree  to  which  the  behavioral  objectives  and  performance  standards  are  met. 

Consideration  was  given  to  the  fact  that  there  would  be  two  users  of  measurement  information  in  the 
ASPT— the  researcher  and  the  instructor  pilot.  For  the  researcher,  it  is  necessary  that  measures  are  of 
sufficient  sensitivity  for  evaluating  relatively  small  effects.  For  the  instructor  it  is  necessary 
that  measures  are  meaningful  and  can  be  readily  interpreted  to  trainees,  and  should  be  designed  to 
provide  diagnostic  information. 

Another  requirement  was  simplicity.  Most  flight  simulation  devices  output  a  relatively  large  number 
of  aircraft  state  and  control  input  parameters  at  a  variety  of  sampling  rates.  Although  there  is  a 
testation  to  use  all  the  data  available,  criterion  performance  should  be  defined  only  on  those 
parameters  which  are  critical  to  the  successful  execution  of  a  maneuver.  A  parameter  should  be  selected 
only  if  it  is  an  essential  component  of  a  maneuver  or  if  it  has  diagnostic  or  feedback  value. 

One  additional  requirement  was  that  the  measurement  system  should  evaluate  performance  on  a 
real-time  basis.  Diagnostic  feedback  is  most  effective  when  provided  immediately  after  execution  of  a 
maneuver.  To  require  extensive  off-line  processing  of  the  data  to  arrive  at  performance  measures  would 
be  unrealistic,  except  for  the  development  of  measures  for  hardware  research.  The  necessity  of 
real-time  measurement  further  eiqphaslzes  the  need  for  simplicity  in  developing  measures  of  proficiency. 

The  focus  of  the  ASPT  APM  system  development  effort  was  the  individual  flight  maneuver.  Within  UPT, 
the  individual  maneuver  represents  the  most  fundamental  unit  of  instruction.  The  intent  was  to  select 
representative  maneuvers  from  all  phases  of  T-37  training  thereby  providing  a  measurement  capability  on 
a  continuum  from  the  simplest  to  the  most  complex  tasks.  The  measurement  development  effort  began  with 
simpler  tasks  and  progressed  through  more  conplex  maneuvers.  To  date,  the  following  scenarios  have  been 
implemented  on  the  ASPT:  (1)  Transition  Tasks  -  Straight  and  Level  Flight,  Airspeed  Changes,  Turns, 
Climbs/Descents;  (2)  Takeoff /Approach/Landing  Tasks  -  Takeoffs,  Tech  Order  Climbs,  Slow  Flight, 
Configuration  Changes,  Straight-in  Approaches,  Overhead  Patterns,  Touch-and-Go's;  (3)  Instrument  Tasks 
-  Rate  Climbs/  Descents,  Vertical  S-A,  Vertical  S-D,  GCA,  Proceed  Direct  to  VOR:  (<)  Aerobatics  - 
Aileron  Rolls,  Barrel  Roll,  Loop,  Split  S,  Cloverleaf,  Cuban  8,  Lazy  8;  and  (5)  Formation  -  Fingertip. 

To  guide  the  definition  of  candidate  performance  measures,  it  was  assjned  that  superior  flying 
performance  in  the  aircraft  or  the  simulator  has  several  characteristics  which  are  reflected  by 
available  flight  parameters.  These  include:  (1)  maintaining  certain  aircraft  state  parameters  close  to 
some  defined  criterion  value;  (2)  avoiding  excessive  rates  and  acceleration  forces  so  that  the  maneuver 


is  executed  smoothly;  (3)  acconplishing  these  objectives  with  the  least  amount  of  effort;  that  Is,  by 
minimizing  control  inputs;  and  (4)  not  exceeding  procedural  or  safety  limits  established  for  the 
maneuver.  For  each  of  these  characteristics,  a  candidate  set  of  measures  was  defined. 

Most  maneuvers  may  be  broken  down  into  segments  for  measurement  purposes.  During  each  segment, 
certain  aircraft  state  parameters  should  be  held  close  to  some  ideal,  or  criterion  value.  The  amount  of 
deviation  from  these  ideal  values  provides  an  index  of  performance.  For  example,  a  simple  turn  to  a 
heading  may  be  broken  into  three  steady  state  segnents.  In  the  first  segment  heading,  altitude  and 
airspeed  are  the  steady  state  parameters  and  deviations  are  measured  from  the  criterion  values.  During 
the  turn,  altitude,  airspeed  and,  bank  are  the  steady  state  parameters.  After  rolling  out  of  the  turn, 
altitude,  airspeed,  and  heading  are  the  steady  state  parameters  again,  but  now  a  new  criterion  value  is 
established  for  heading. 

The  most  common  state  parameters  measured  are  altitude,  airspeed,  heading,  and  bank.  However, 
complex  maneuvers  occasionally  contain  other  parameters  which  should  be  held  constant  during  part  or  all 
of  the  maneuver.  These  maneuvers  usually  require  that  a  new  state  parameter  be  computed  and  the 
deviation  be  measured  from  the  computed  value.  For  example,  during  a  traffic  pattern,  the  pilot  should 
be  able  to  determine  and  maintain  an  angle  of  bank  in  the  final  turn  which  will  enable  proper  runway 
alignment  when  he  rolls  out.  The  required  bank  in  this  case  must  be  continuously  computed  using  the 
current  aircraft  position  and  heading.  The  bank  deviation  is  then  confuted  by  comparing  the  actual  bank 
to  the  computed  ideal  value. 

Although  deviations  from  desired  values  provide  an  index  of  the  amount  of  error  at  any  one  instant, 
it  is  necessary  to  summarize  the  information.  For  each  parameter,  both  the  arithmetic  mean  deviation 
and  the  root-mean-square  (RMS)  deviation  are  computed.  In  addition,  a  tolerance  band  is  set  for  each 
steady  state  parameter.  The  percent  of  time  during  the  maneuver  that  the  deviation  is  above  the  toler¬ 
ance,  within  tolerance,  or  below  tolerance  is  confuted.  These  time  on  tolerance  measures  were  designed 
primarily  for  student  feedback.  Two  other  measures  are  also  computed  which  have  often  been  used  in 
previous  pilot  performance  data  collection,  the  maximum  and  minimum  values  for  each  state  parameter. 
Aside  from  these  measures  continuously  computed  over  some  portion  of  the  maneuver,  single  values  are 
also  recorded  at  key  points  for  certain  maneuvers.  For  example,  speed  at  rotation  and  speed  at  gear 
retraction  are  recorded  for  the  takeoff.  Since  these  values  are  dependent  upon  the  specific  maneuver, 
no  coninon  set  of  measures  could  be  defined. 

While  the  state  parameter  deviations  are  the  primary  measure  of  performance,  certain  other  measures 
are  computed  which  reflect  how  smoothly  the  maneuver  is  executed.  They  describe  the  rates  and 
accelerations  of  the  simulated  aircraft  about  the  vertical,  longitudinal  and  lateral  axes.  Pitch,  roll 
and  heave  were  chosen  since  preliminary  data  Indicated  these  axes  to  be  the  only  ones  delivering 
perceptible  force  cuing  information.  RMS  rates  and  accelerations  are  computed  for  these  axes  to  furnish 
smoothness  measures. 

The  effort  expended  by  the  pilot  may  be  determined  by  characteristics  of  the  forces  exerted  on  the 
controls  and  the  distances  the  controls  are  moved.  Five  primary  flight  controls  were  investigated:  (1) 
elevator  (Y-axis);  (2)  aileron  (X-axis);  (3)  rudder;  (4)  throttle;  and  (5)  trim.  Since  the  stick  was 
considered  the  primary  flight  control,  most  measures  were  defined  to  characterize  its  movement.  For 
elevator  and  aileron  control,  four  measures  were  defined:  (1)  RMS  position  (deviation  from  zero  point); 
(2)  RMS  movement;  (3)  RMS  power  (force  times  movement);  and  (4)  number  of  reversals.  For  the  elevator, 
both  mean  force  and  RMS  force  were  also  considered  of  interest.  For  rudder  control,  only  two  measures 
were  defined:  (1)  RMS  power;  and  (2)  RMS  movement.  For  throttle  control,  only  one  measure,  RMS 
movement  was  defined.  For  trim  control,  one  measure  was  defined,  the  percentage  of  time  elevator  force 
remained  within  some  tolerance  band. 

Certain  maneuvers  require  that  the  pilot  perform  some  procedures  in  a  specified  time  Interval  during 
the  maneuver  or  that  he  maintain  the  aircraft  within  certain  safety  limits.  The  traffic  pattern  is  a 
good  example  of  this  type  of  maneuver.  The  pilot  must  lower  the  speedbrake,  landing  gear  and  flaps  at 
specified  times  during  the  approach.  These  types  of  procedures  may  be  monitored  in  the  APM  system  and 
logicals  set  true  or  false,  denoting  whether  or  not  the  procedure  was  accomplished  in  the  appropriate 
time  interval.  In  addition,  certain  safety  limits  have  been  established  for  the  complex  maneuvers.  In 
the  traffic  pattern,  an  error  logical  is  set  if  the  final  approach  is  too  low  or  too  slow  or  if 
touchdown  occurs  at  some  place  other  than  the  prescribed  area  on  the  runway.  Since  such  errors  are 
completely  dependent  upon  the  specific  maneuver,  no  common  set  of  measures  could  be  defined. 

Although  maneuver  execution  is  a  continuous  process,  it  may  be  conceptualized  as  Integrated 
sequences  of  steady  states  and  transitions.  The  fundamental  flight  attitudes  plus  transitions  from  one 
attitude  to  another  form  the  conceptual  segnents  for  most  maneuvers  (74).  In  the  30°  turn  to  heading, 
prior  to  the  roll-in  and  after  the  roll-out,  the  desired  angle  of  bank  is  zero.  During  the  turn, 
however,  the  desired  value  is  30°.  For  the  purpose  of  measuring  deviation  from  desired  bank  angle,  it 
is  easier  to  divide  the  turn  Into  three  segments  and  measure  the  difference  against  a  constant  value  for 
each  sepient  than  to  generate  a  continuous  function  for  the  entire  maneuver.  Such  maneuver  secynentation 
has  been  utilized  In  most  previous  efforts  (75,  76,  77). 

While  the  segmentation  approach  appears  straightforward,  two  problems  can  occur— the  definition  of 
the  start/stop  logic  rules  and  the  measurement  of  transitions.  In  the  turn-to-headlng  example,  when  Is 
It  appropriate  to  start  measuring  deviations  from  the  desired  30°  angle  of  bank?  The  approach  used  In 
the  present  effort  was  to  Initiate  a  timer  once  a  certain  condition  had  been  met  and  begin  measurement 
once  a  certain  amount  of  time  had  elapsed.  In  the  turn-to-headlng  scenario,  the  timer  was  Initiated 
whenever  bank  angle  was  greater  than  15°.  After  three  seconds,  deviations  from  the  desired  30°  bank 
angle  are  scored.  The  delay  time  was  based  on  the  performance  of  experienced  pilots.  In  this  manner, 
scoring  1$  Initiated  whenever  the  pilot  should  have  achieved  the  desired  bank  angle. 
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For  other  maneuvers,  start/stop  logic  rules  were  based  on  published  Air  Training  Command  (ATC) 
criteria.  For  example,  start/stop  logic  for  the  climb  used  the  rule  that  altitude  lead  point  for 
level-off  from  a  climb  should  be  10*  of  vertical  velocity.  In  the  steep  turn,  deviation  from  desired 
bank  angle  was  computed  until  the  command,  "roll-out"  was  given.  In  each  case,  the  key  ingredient  was 
that  the  logic  rules  would  unequivocally  determine  that  a  particular  segment  had  been  entered  or 
departed.  The  same  logic  approach  was  used  to  determine  when  to  measure  specific  values  such  as 
rotation  speed  or  vertical  velocity  at  touchdown. 

The  second  difficulty,  the  measurement  of  transitions,  presents  even  greater  problems  for  developing 
adequate  measurements.  First,  some  transition  segments  are  relatively  brief  In  duration.  Thus,  very 
little  data  can  be  obtained.  Second,  there  are  no  readily  defined  criterion  referenced  objectives  for 
these  transition  segnents.  And  third,  it  is  unclear  how  much  performance  during  these  transitions 
contributes  to  overall  proficiency  for  the  maneuver.  Consequently,  it  was  decided  not  to  provide 
specific  measurement  for  the  individual  transitions  with  the  exception  of  those  parameters  which  should 
be  held  constant  (e.g.,  airspeed  and  altitude  during  a  roll-in). 

To  develop  sumnary  assessment  scores,  a  sample  of  ten  experienced  IP's  flew  five  repetitions  of  a 
maneuver.  Descriptive  statistics  on  RMS  error  for  each  parameter  were  computed  and  confidence  Intervals 
established  such  that  experienced  pilots  could  be  expected  to  stay  within  these  limits  80X  of  the  time. 
These  limits  were  then  used  as  tolerance  bands  for  confuting  percentages  of  time  above,  within,  or  below 
limits  and  an  overall  tlme-on-target  (TOT)  score  is  computed  as  the  maneuver  progresses.  This  score  Is 
the  percent  of  time  all  appropriate  state  parameters  are  simultaneously  within  tolerance.  If  one  or  more 
state  parameter  moves  out  of  tolerance,  the  TOT  score  will  not  Increase.  The  score  will  begin  to  in¬ 
crease  again  only  after  all  parameters  are  back  in  tolerance.  Maneuver  secpnents  were  weighed  according 
to  their  perceived  inportance  by  experienced  IP's  and  the  weights  used  In  determining  the  summary  scores. 

Each  maneuver  has  a  unique  CRT  display  format.  The  display  may  be  generated  on  an  In-cockpit  CRT 
for  student  feedback  and  may  be  copied  for  use  later  in  a  debriefing.  The  display  is  designed  to 
include  alphanumeric  titles  and  selected  parameters  available  In  the  computer  math  models  or  In 
preprogramming.  The  percent  high,  on,  and  low  scores,  as  well  as  the  total  score  is  displayed  for  each 
maneuver.  In  addition,  error  messages  or  other  Information  may  be  displayed,  depending  on  the 
particular  maneuver. 

A  Student  Data  System  (SDS)  was  developed  for  the  storage  of  data  collected  during  each  exercise 
se^nent.  Certain  Identification  information  also  Is  stored  as  part  of  the  segment  data  record.  Some  of 
the  Identifier  Information  is  manually  input  to  the  SDS  and  the  remainder  is  automatically  Input  from 
parameters  available  In  the  computer  programs.  The  identifier  Information,  primary  performance 
measures,  secondary  performance  measures,  TOT  scores,  and  error  messages  are  transmitted  for  storage  to 
a  disc  file  inmediately  after  a  maneuver  Is  completed.  The  data  record  Is  also  displayed  on  a  CRT  at 
the  console  and  output  on  a  line  printer  for  examination. 

To  minimize  the  requirements  for  performance  data  storage,  it  was  necessary  to  develop  a  generalized 
retrieval  system  which  could  sort  and  perform  some  statistical  analysis  of  data  stored  in  the  SDS.  The 
present  system  is  an  off-line,  batch-type  program  which  accepts  data  cards  as  inputs  to  define  the  data 
to  be  returned  and  analyzed.  The  retrieval  and  analysis  program  allows  the  researcher  to  make  a 
thorough  Inspection  or  preliminary  analysis  of  the  data  while  a  project  is  underway  or  after  It  Is 
conpleted.  Other  analysis  routines  can  be  added  to  the  program  to  fit  the  requirements  of  a  particular 
research  design. 

The  ASPT  preprograming  system  provided  the  basic  framework  for  the  APM  system  wherein  FORTRAN 
programs  may  be  included  in  the  ASPT  software.  These  programs  can  access  all  parameters  used  In  the 
flight  simulation  and  perform  confutation  In  real  time,  as  the  simulator  Is  being  flown.  The  basic  units 
of  the  preprograming  system  are  the  exercise  segments,  complete  programs  designed  to  measure  individual 
maneuvers.  Each  segnent  Is  conposed  of  up  to  16  separate  cases. 

The  first  case  in  each  exercise  segment,  the  Initialization  case,  sets  the  simulator  to  the  Initial 
conditions  selected  for  the  maneuver.  Intermediate  cases  contain  the  scoring  logic,  which  determines 
the  parameters  to  be  measured  during  the  maneuver.  When  certain  conditional  statements  In  the  program 
are  satisfied,  selected  messages,  composed  of  any  of  189  words,  are  transmitted  through  the 
communication  system.  These  messages  notify  the  pilot  when  to  start  the  maneuver,  provide  him 
Information  during  certain  maneuvers,  and  notify  him  with  a  tone  when  maneuver  scoring  Is  complete. 

The  final  case  In  each  exercise  segment  Is  the  endpoint  case.  When  certain  conditions  are  met  which 
signify  that  the  maneuver  Is  complete,  the  simulator  automatically  freezes.  Up  to  12  exercise  se9nents 
may  be  grouped  into  a  single  exercise.  This  allows  efficient  sequencing  from  one  maneuver  to  the  next. 
When  a  maneuver  terminates  and  automatlcal-ly  freezes,  the  operator  may  manually  unfreeze  the  simulator. 
This  will  automatically  sequence  It  to  the  next  exercise  sequent  and  the  simulator  will  Initialize  for 
the  next  maneuver. 

MEASUREMENT  SYSTEM  DEVELOPMENT  FOR  THE  C-5A  OPERATIONAL  FLIGHT  SIMULATOR  (OFS) 

Traditionally,  aircrew  performance  evaluation  In  the  USAF  has  consisted  of  standardization  and 
evaluation  (STAN/EVAL)  checkrldes  administered  by  highly  qualified  flight  examiners.  Such  subjective 
techniques  have  been  used  extensively  in  all  phases  of  flight  training.  While  they  are  generally 
adequate  for  certifying  Initial  qualification,  these  techniques  possess  Inherent  weaknesses  which  limit 
their  usefulness  In  certain  applications.  Because  of  such  limitations,  operational  command  training 
managers  realized  their  need  for  proved  performance  measurement  capabilities.  In  1976,  the  Air  Force 
Hirnn  Resources  Laboratory  and  Military  Airlift  Comaand  (MAC)  Initiated  a  cooperative  effort  to:  (1) 
Identify  the  mission  task  requirements  for  MAC  aircrew  members  through  an  analysis  of  training 
syllabuses.  STAN/EVAl  performance  criteria,  and  flying  training  records;  (2)  describe  current  and 
planned  flight  simulator  and  aircraft  capabilities  in  terms  of  their  use  In  generating  mission 
performance  measurement  for  MAC  aircrew  members;  and  (3)  develop  and  evaluate  an  objective/quantitative 


26 


measurement  system  for  MAC  aircrews  based  on  the  mission  task  requirements  and  the  capabilities  of  the 
Comnand's  flight  simulation  systems  as  well  as  Its  aircraft.  The  requirements  study  for  this  effort  Is 
described  In  Swlnk,  Butler,  Lankford,  Miller,  Watkins,  and  Waag  (78).  This  APM  system  development 
effort  is  currently  In  progress. 

In  contrast  to  the  research  orientation  of  the  ASPT  APM  system  development,  the  goal  of  the  C-5  APM 
system  development  effort  Is  to  determine  the  operational  suitability  and  utility  of  an  APM  system 
within  the  training  program.  Because  of  the  training  orientation,  it  was  readily  apparent  that  the 
measurement  system  should  fulfill  certain  requirements.  Since  the  C-5  Is  a  multi-crewed  aircraft.  It 
was  necessary  to  assess  proficiency  for  each  crevrni ember  in  the  performance  of  his  assigned  tasks  as  well 
as  his  performance  in  the  coordination  among  crewmembers,  the  pilot,  copilot,  and  flight  engineer 
stations.  The  loadmaster  was  not  included  since  coordinated  training  In  the  flight  simulator  Is  not 
provided  for  that  crew  position. 

The  original  concept  was  that  the  C-5  APM  system  would  be  used  to  evaluate  and  document  aircrew 
proficiency  at  the  end  of  the  training  program.  In  this  case,  it  was  desirable  that  the  APM  system 
function  in  an  automated,  hands-off  mode  thereby  ensuring  standardization,  reducing  the  instructor's 
workload  and  eliminating  the  possibility  of  manual  record  keeping  errors.  Since  measurement  is  an 
integral  part  of  training  as  well  as  evaluation,  potential  applications  to  simulator  training  sorties 
were  considered.  This  led  to  the  requirement  for  some  degree  of  instructor  intervention  and  control  of 
the  mission.  Thus  the  instructor  was  provided  the  capability  to  alter  environmental  conditions,  insert 
malfunctions,  select  different  profiles  for  scoring  and  select  alternative  feedback  displays.  Another 
requirement  was  that  the  C-5  APM  system  provide  real-time  aircrew  performance  feedback  to  the 
instructor.  A  further  feedback  requirement  was  that  the  performance  evaluation  information  be 
summarized  and  a  hardcopy  report  be  printed  for  debriefing  purposes  and  further  analyses  after  the 
training  session. 

Whereas  the  focus  of  the  ASPT  APM  system  was  the  individual  flight  maneuver,  the  C-5  APM  system 
concentrated  on  the  development  of  full  mission  profiles.  Consequently,  a  C-5  simulator  evaluation 
mission  profile  was  constructed  which  represented  a  typical  MAC  airlift  mission.  Through  interview  and 
discussion  with  MAC  training  personnel,  review  of  C-5  training  documents,  and  study  of  the  C-5  Flight 
Manual,  the  representative  mission  segments  were  determined.  The  resulting  mission  profile  was  designed 
to  contain  the  activities  required  of  the  various  crewmembers  while  performing  their  normal  inflight 
duties.  The  mission  also  included  contingencies  and  emergency  situations  which  require  crew 
coordination  among  various  aircrew  members  and  which  are  typical  of  operational  mission 
malfunction/emergencies.  Although  development  of  a  nominal  mission  profile  provided  an  excellent 
vehicle  for  defining  a  candidate  set  of  performance  measures,  its  usefulness  in  training  would  be 
severely  limited.  For  this  reason,  the  C-5  APM  system  was  designed  to  provide  the  capability  to 
generate  new  mission  profiles  in  an  off-line  mode.  In  this  manner,  mission  scenarios  could  be  developed 
for  a  variety  of  training  situations. 

Three  categories  of  tasks  were  defined  for  implementation  in  the  C-5  APM  system:  (1)  checklists  and 
procedures;  (2)  navigational  profiles;  and  (3)  parameter  control.  For  each  of  these  categories,  a 
candidate  set  of  measures  were  defined.  Since  operation  of  the  C-5A  is  highly  procedural ized,  the 
measurement  of  the  accuracy  of  checklist  procedures  performance  represents  a  major  task  of  the 
measurement  system.  The  C-5  APM  system  differentiates  among  five  types  of  checklists  and  procedures: 
(1)  normal  checklists;  (2)  emergency  checklists;  (3)  normal  procedures;  (4)  minor  emergency  procedures; 
and  (5)  major  emergency  procedues.  Each  checklist  or  procedure  is  comprised  of  a  series  of  individual 
steps.  Each  step  or  block  of  steps  may  be  designated  as  sequential  or  non-sequential.  Non- sequential 
steps  may  be  performed  in  any  order  while  sequential  steps  require  a  definite  order  of  occurrence. 
Three  types  of  procedural  errors  are  possible:  (1)  an  omission  error  in  which  the  step  is  either  not 
accomplished  or  not  completed  within  time  allowances;  (2)  a  constraint  error  in  which  a  step  is 
accomplished  under  inappropriate  conditions;  and  (3)  a  sequence  error  in  which  the  step  is  not  performed 
in  the  correct  order. 

The  C-5  APM  system  provides  for  real-time  assessment  of  navigation  profiles  including:  (1)  takeoffs 
and  instrument  departures;  (2)  enroute;  (3)  holding  patterns;  (4)  initial  approach  and  non-precision 
final;  and  (5)  ILS  final  approach  and  landing.  Measured  parameters  consist  of  those  relating  to  the 
maintenance  of  ground  track  and  adherence  to  altitude  restrictions.  Computed  measures  include  RMS 
deviation  as  well  as  time-within- tolerance  percentages.  The  C-5  APM  System  also  provides  the  capability 
to  measure  the  degree  to  which  certain  parameters  are  held  constant  or  maintained  within  certain 
limits.  Each  parameter  may  have  multiple  envelopes;  for  example,  desired  airspeed  will  vary  according 
to  the  phase  of  flight.  Computed  measures  include  mean  deviation  and  RMS  deviation.  Upper  and  lower- 
limits  can  be  established  so  that  time-on-tolerance  measures  can  be  computed. 

The  start-stop  logic  rules  for  measuring  aircrew  procedures  in  the  C-5  APM  system  are  similar  to 
those  used  in  the  ASPT.  Each  start/stop  criterion  consists  of  from  one  to  five  mathematical  logical 
expressions,  combined  by  a  series  of  logical  AND,  ORs  and  NOTs.  These  conditions  also  may  be  grouped 
using  parentheses.  The  problems  associated  with  precisely  defining  these  criteria  are  similar  to  those 
encountered  in  the  ASPT  development.  This  is  especially  true  with  respect  to  the  initiation  of 
automated  procedures  or  checklist  monitoring.  The  knowledgeable  procedure  writer  will  insure  that 
procedure  monitoring  can  begin  as  soon  as  possible.  These  scenarios  must  be  considered  limitations  in 
the  totally  automated  performance  monitoring  of  the  APM  system.  However,  in  the  manual  mode,  the 
instructor  always  has  the  option  of  starting  the  monitoring  of  a  procedure  in  advance  of  the  malfunction 
activation.  Thus,  the  C-5  APM  system  will  be  capable  of  providing  maximum  feedback  concerning  a 
procedure,  regardless  of  such  factors  as  the  generality  of  the  malfunction  start  conditions. 

A  consequence  of  developing  full-mission  profiles  1$  that  the  number  of  individual  tasks  is  large 
and  the  number  of  performance  measures  even  larger.  Consequently,  it  is  necessary  to  summarize  the 
data.  To  do  so,  different  levels  of  assessment  have  been  defined.  At  the  most  elemental  level, 
performance  measures  for  a  single  task  can  be  sumaarized  as  a  single  score.  At  the  next  level. 
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individual  scores  for  all  tasks  belonging  to  the  same  group  Mill  be  combined  to  form  a  single  score.  At 
the  next  level,  each  crew  member  receives  a  sjimary  score  for  each  group  of  tasks.  At  the  highest 
level,  these  scores  are  combined  to  form  an  aircrew  composite  score  for  the  entire  mission. 

To  arrive  at  summary  scores,  a  point  system  Is  used.  Each  measure  within  a  given  task  is  assigned  a 
possible  number  of  points.  This  reflects  the  relative  inportance  of  each  measure  within  the  task. 
Furthermore,  rules  are  established  for  defining  the  number  of  earned  points  for  a  given  performance.  A 
critical  weighting  is  associated  with  each  measure  which  reflects  the  seriousness  of  the  error  In  the 
event  a  certain  level  is  not  achieved.  The  earned  points  are  sunned  and  then  reduced  according  to  the 
criticality  factors.  The  reduced  earned  points  are  divided  by  the  possible  points  to  produce  a 
percentage.  Initially,  the  point  values  and  criticality  weights  to  be  Implemented  in  the  C-5  A PM  system 
will  be  the  result  of  subjective  Instructor  judgments.  This  approach  will  be  used  for  the  development 
of  sunmary  scores  at  each  level  of  assessment. 

Performance  feedback  during  a  mission  will  be  provided  by  two  CRT  displays  located  in  the  cockpit. 
One  will  be  used  by  the  instructor  pilot  and  the  other  by  the  Instructor  flight  engineer.  A  line  printer 
will  generate  a  hard-copy  debriefing  report.  Seven  categories  of  displays  and  feedback  will  be 
included:  (1)  Mission  sequence:  sunmary  displays  of  the  sequences  of  tasks;  (2)  Mission  plot:  graphic 
presentation  of  aircraft  progress  with  reference  to  ground  track;  (3)  Route  chart:  graphic  background 
displays  corresponding  to  departure,  enroute  and  approach  plates;  (4)  Checklist/procedure:  displays  of 
pre-defined  sequences  of  actions  to  be  performed  by  crewmembers;  (5)  Error  alert:  message  alerting 
instructor  to  crew  errors  as  they  occur  in  the  pre-defined  tasks;  (6)  Proficiency  assessment:  detailed 
alphanumeric  displays  relative  to  any  specific  pre-defined  performance  segment  or  task;  and  (7) 
Debriefing  report:  hard-copy,  objective  performance  data  with  which  the  instructor  may  assess  and 
evaluate  performance. 

During  the  course  of  a  mission  the  C-5  APM  system  software  will  generate  a  file  containing  all  data 
necessary  for  computing  the  overall  assessment  scores  and  production  of  the  final  debriefing  report. 
All  proficiency  assessment  scores  will  be  stored  on  a  disk  cartridge  for  later  retrieval.  Editing 
capabilities  will  be  provided  so  that  only  the  desired  data  are  retrieved.  A  statistical  package  also 
will  be  included  so  that  the  retrieved  data  may  be  analyzed.  This  analysis  package  was  Included  in  the 
prototype  system  to  aid  in  follow-on  test  and  evaluation  of  the  C-5  APM  system. 

In  the  initial  requirements  study,  the  C-5  Operational  Flight  Simulator  (OFS)  was  Investigated  to 
determine  whether  its  present  configuration  would  support  Implementation  of  an  APM  system.  Because  of 
limitations  in  the  C-5  OFS  computational  system,  it  was  decided  that  such  an  approach  would  not  be 
feasible.  It  was  concluded  that  the  most  cost-effective  approach  would  be  to  aucpnent  the  existing  C-5 
OFS  by  means  of  an  autonomous  strap-on  system  which  would  ride  "piggy-back"  on  the  basic  simulation 
system.  The  approach  would  insure  that  the  functional  and  physical  characteristics  of  the  C-5  OFS 
remain  intact.  Such  a  system  enploys  a  mini-computer  and  requires  the  development  of  a  trainer 
Interface  which  allows  the  system  to  capture  all  necessary  data  and  to  provide  the  necessary  control 
over  the  host  simulator. 

The  configuration  of  the  C-5  APM  system  consists  of  three  major  functional  groups.  The  first  group 
consists  of  the  operator  interface  hardware,  a  combined  keyboard/CRT  display  in  the  Radio  Aids  Station, 
referred  to  as  the  System  Control  Terminal.  The  terminal,  plus  a  line  printer  unit,  will  provide  for  APM 
operations  control  and  documentation.  Additionally,  on  the  flight  deck,  displays  and  keyboards  will  be 
provided  for  the  instructor  pilot  and  for  the  Instructor  flight  engineer  to  enhance  the  real-time 
feedback  to  the  Instructors.  Using  their  keyboards.  Instructors  may  select  various  displays  of 
information  on  their  monitors,  as  well'  as  control  various  aspects  of  the  mission  scenario.  The  second 
group  consists  of  the  trainer  interface  hardware  with  which  two  hardware  Interface  units  will  acquire 
data  from  the  C-5  OFS  and  control  environmental  and  malfunction  factors  as  related  to  the  content  of 
each  mission  scenario.  The  third  functional  group  consists  of  the  system  control  hardware,  the  major 
component  of  which  Is  a  fast,  general  purpose  mini-computer.  Software  will  control  and  Implement  all 
APM  monitoring  through  an  Interface  to  disk  and  diskette  mass  storage  units  associated  with  the 
mini-computer. 

APM  SYSTEM  VALIDATION 

The  most  critical  characteristic  of  any  measure  Is  Its  validity.  For  most  complex  tasks,  there  Is 
no  single  necessary  and  sufficient  test  that  can  be  applied  to  candidate  measures  to  assess  their 
validity.  Therefore,  several  validation  tests  must  be  applied  during  the  course  of  a  measurement 
development  effort.  The  type  of  validity  most  appropriate  for  the  development  of  measures  of  pilot 
performance  is  content  validity.  It  demands  that  the  most  salient  behavioral  components  be  Incorporated 
Into  the  measure.  To  the  extent  that  the  pilot  performance  measure  addresses  all  relevant 
criterion-referenced  objectives,  the  content  validity  will  be  high.  It  is  clear  that  there  is  a 
relationship  between  the  adequacy  of  the  task  definition  and  the  validity  of  the  resulting 
measurements.  By  taking  the  criterion-referenced  approach  in  the  development  of  performance  measures  in 
the  ASPT  and  the  C-5  simulator,  a  satisfactory  degree  of  content  validity  was  established. 

Aside  from  content  validity,  measures  should  possess  eq>1r1cal  validity.  For  objective  measures  of 
pilot  performance,  there  are  at  least  four  criteria  by  which  empirical  validity  may  be  established. 
First,  the  measures  should  successfully  discriminate  among  pilots  of  different  experience  levels,  for 
example,  novice  pilots  vs  instructor  pilots.  Second,  these  measures  should  be  positively  correlated 
with  concurrent  measures  of  performance  such  as  IP  evluatlons.  Third,  they  should  be  sensitive  to  the 
effects  of  training.  And  fourth,  objective  measures  of  performance  should  be  sensitive  to  performance 
decrements  resulting  from  adverse  environmental  or  pilot  stress  factors. 

Unfortunately,  no  large  scale  validation  study  of  the  ASPT  measurement  system  has  been 
•ccoasil Ished.  Nonetheless,  data  collected  within  the  context  orf  specific  research  studies  have  provided 
some  evidence  of  the  empirical  validity  of  the  system.  Haag,  Eddowes,  Fuller,  and  Fuller  (79)  found 
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that  the  objective  measures  for  six  of  the  basic  transition  tasks  flown  under  instrunent  conditions 
successfully  discriminated  between  novice  and  experienced  pilots.  Furthermore,  significant  correlations 
between  the  objective  measures  and  IP  ratings  were  obtained. 

In  a  study  addressing  the  contributions  of  simulated  platform  motion  to  training  effectiveness, 
Martin  and  Uaag  (43)  used  a  unit  weighting  procedure  to  develop  a  single  score  for  basic  transition  and 
takeoff/ landing  tasks.  Using  these  scores,  significant  learning  effects  were  demonstrated  during 
simulator  training  for  straight-and- level,  airspeed  changes,  climbs/descents,  slow  flight,  takeoffs, 
straight-in  approaches/landings,  and  overhead  patterns. 

Nataupsky  et  al.  (45)  analyzed  certain  of  the  Individual  measures  within  a  maneuver,  and  obtained 
significant  learning  effects  for  takeoffs,  steep  turns,  slow  flight,  and  stralght-ln  approaches.  In 
this  study,  moderate  correlations  between  overall  IP  ratings  and  the  total  score  currently  computed  In 
the  APM  system  were  obtained. 

Irish  et  al.  (28)  and  Irish  and  Buckland  (29)  studied  the  effects  of  various  simulator 
configurations  on  the  performance  of  experienced  pilots.  Two  of  the  conditions  involved  various  levels 
of  turbulence  and  celling/visibility.  Degraded  performance  as  a  result  of  these  two  adverse 
environmental  conditions  were  reflected  by  the  objective  scores  from  the  APM  system  and  furnish 
additional  evidence  of  Its  empirical  validity. 

Despite  the  fact  that  a  large  scale  validation  study  has  not  been  accomplished,  the  studies  reviewed 
above  provide  evidence  of  the  empirical  validity  of  the  ASPT  APM  system.  Until  an  overall  effort  Is 
completed,  however,  the  ASPT  and  C-5  APM  system  should  be  considered  as  candidate  sets  of  measures. 
Clearly,  there  Is  a  need  for  further  validation  efforts  of  the  ASPT  and  C-5  APM  systems. 


IX.  FUTURE  SIMULATOR  TRAINING  DEVELOPMENTS 


PROJECTIONS 

To  complete  this  description  of  the  use  of  simulators  for  training  In-flight  and  emergency 
procedures,  it  may  be  useful  to  project  current  programs  Into  the  future  and  generate  a  preview  of 
future  simulator  flying  training  developments.  Taking  Into  account  the  Increasing  seriousness  of  the 
diminishing  availability  of  energy  resources.  It  is  reasonable  to  forecast  an  .  Increase  In  the  rate  of 
exploitation  of  the  capabilities  of  modern  high  performance  flight  simulators  In  future  flying  training 
programs.  If  flying  continues  to  be  a  viable  means  of  transportation  and  an  Integral  part  of  world 
military  forces  In  the  future.  It  Is  Inevitable  that  the  use  of  flight  simulators  will  Increase.  In 
addition,  it  Is  likely  that  the  routine  use  of  simulators  with  visual  systems  capable  of  supporting 
training  In  contact  maneuvers  will  Increase  dramatically.  Further,  as  more  modern  simulation  systems 
enter  service,  training  managers  will  discover  new  ways  to  use  them,  such  as  In  military  tactical  and 
continuation  training  applications. 

Today's  high  performance  simulators.  Including  those  equipped  with  visual  displays,  often  have  been 
employed  mainly  for  the  Initial  training  of  beginning  student  pilots.  Many  airline  training  programs, 
however,  have  successfully  demonstrated  application  of  flight  simulators  in  upgrade  and  in  periodic 
certification  training  programs  In  which  the  trainees  are  experienced  aircrew  personnel.  Extrapolating 
from  these  proven  applications  and  considering  the  costs  of  continuation  training.  It  may  be  decided 
that  simulators  should  take  over  a  share  of  the  requirements  for  maintaining  aircrew  combat  readiness. 
Should  this  occur.  It  can  be  seen  that  tomorrow's  simulators  may  be  designed  with  capabilities  for 
supporting  such  continuation  training  requirements  as  multi-ship  tactical  strike  missions  or  air  combat 
maneuvering  training  In  a  variety  of  types  of  practice  engagements.  In  other  words,  future  flight 
simulators  will  not  only  Involve  the  basic  training  of  Inexperienced  pilots,  but  will  take  over  portions 
of  the  bombing  and  air  combat  maneuvering  training  of  fully-qualified,  combat-ready  aircrews.  One 
further  development  can  be  forecast  given  development  of  such  flight  tactics  simulators.  These  new 
devices  may  be  located  at  different  sites  with  their  functions  Interconnected  and  Integrated  to  provide 
extensive  real-time  combat  training  exercises.  Such  a  development  would  represent  an  electronic  version 
of  the  USAF's  current  Red  Flag  tactics  training  program  (80,  81). 

In  this  kind  of  future  flying  training  development,  the  form  of  the  engagement  simulation  system 
probably  will  be  vastly  different  from  that  of  current  self-contained  flight  simulators.  Studies 
reviewed  elsewhere  In  this  report  suggest  that  realism  and  fidelity  requirements  have  been  routinely 
exceeded  In  state  of  the  art  simulators.  Future  research  may  be  expected  to  describe  more  precisely 
how,  how  closely  and  in  what  way,  the  flight  simulator  must  match  the  aircraft  It  simulates  to  be  an 
effective,  efficient  and  economical  training  aid.  Meanwhile,  a  prediction  appears  warranted  that  future 
trainee  stations  In  the  kinds  of  tactics  simulation  systems  discussed  above  will  be  less  representative 
and  much  more  austere  than  current  simulator  cockpits  and  that  the  complexity  and  elegance  of  the  simu¬ 
lation  model  of  the  combat  environment  will  be  substantially  reduced  from  simulator  math  models  of  today. 

As  Increasing  mmbers  of  flight  simulators  with  greater  performance  capabilities  become  available, 
they  will  undoubtedly  be  applied  more  frequently  to  accoanodate  a  greater  number  of  training 
requirements.  In  addition  to  their  application  in  continuation  training  programs  for  experienced  and 
qualified  aircrew  personnel,  tomorrow's  simulators  also  may  be  expected  to  serve  as  training  tools  to 
support  practice  and  preparation  for  flight  examinations.  In  mission  rehearsal  and  In  periodic  refresher 
training  to  document  aircrew  qualifications. 

The  effectiveness  of  flight  simulators  In  a  broader  arrqy  of  training  applications  will  In  time 
demonstrate  convincingly  their  substantial  effectiveness  and  value  In  flying  training.  Similarly, 
training  managers  will  learn  that  a  much  wider  variety  of  simulators  will  satisfy  their  training 
requirements.  This  may  lead  to  a  requirement  for  more  different  kinds  of  simulators,  each  perhaps  with 


more  specific  training  functions  and  probably  with  more  finely  tailored  and  limited  capabilities.  A 
correlated  decrease  in  simulator  cost  at  no  penalty  In  training  effectiveness  also  may  be  feasible. 

Given  the  viability  of  the  above  prediction,  a  need  will  become  clear  for  an  overall  plan  for 
organizing  and  integrating  a  structured  program  In  which  many  different  flight  simulation  devices 
participate  to  optimize  the  effectiveness  of  the  training  provided.  Solving  the  problem  of  Integrating 
the  various  elements  of  a  comprehensive  flying  training  program  will  be  facilitated  through  the  use  of 
the  performance  measurement  capabilities  being  designed  into  more  and  more  flight  simulators  under 
development  today.  The  performance  measurement  capabilities  of  tomorrow’s  flight  simulators  will 
furnish  training  managers  with  quantitative  indices  of  their  training  effectiveness  which  may  be  used  to 
guide  in  fine-tuning  the  integration  of  the  devices  In  the  training  program.  The  same  performance 
measures  also  may  be  used  to  assess  Individual  progress  and  In  this  application  serve  as  a  basis  for 
optimizing  the  trainee's  readiness  at  each  stage  of  his  training  to  take  maximum  advantage  of  his  next 
learning  experience. 

This  discussion  of  future  simulator  developments  describes  flying  training  programs  substantially 
different  than  many  of  those  of  today.  The  application  of  flight  simulators  in  flying  training  will 
increase,  driven  by  the  scarcity  and  cost  of  fuel.  Simulators  will  be  integrated  Into  initial  and 
continuing  training  programs  for  use  with  Inexperienced  as  well  as  fully-qualified  aircrew  personnel  to 
achieve  an  increasingly  wide  variety  of  training  objectives.  Utilization  rates  of  between  40  and  80 
hours  a  week,  50  weeks  a  year,  can  be  expected  to  realize  the  simulator's  training  cost-effectiveness 
potential.  The  capabilities  of  high  performance  digital  computers  will  be  exploited  and  refined  to 
support  the  Increased  simulator  training  Involvement  In  many  contact  and  tactical  training  areas  and  to 
reveal  and  document  requirements  for  training  environment  fidelity  and  realism.  Finally,  simulator  air¬ 
crew  performance  measurement  capabilities  will  be  developed  and  applied  to  Improve  the  effectiveness  of 
instruction  given  individual  trainees. 

CONCLUSIONS 

In  view  of  the  extent  and  the  limitations  of  information  describing  current  flying  training  tech¬ 
nology,  the  following  conclusions  are  offered,  based  on  the  material  presented  in  this  report: 

1.  The  world's  energy  resource  situation  requires  that  flying  training  be  accomplished  In  new 
ways  which  minimize  use  of  petroleum-based  fuels. 

2.  Flight  simulators  have  proven  to  be  effective  training  aids  In  nearly  every  test  or 
application  In  a  flying  training  program.  In  many  simulator  applications  there  Is  no  compelling 
relationship  between  training  effectiveness  and  fidelity/realism. 

3.  State  of  the  art  flight  simulators  have  the  potential  for  supporting  a  wide  variety  of 
contact  training  requirements  as  well  as  virtually  all  Instrunent  training. 

4.  The  training  effectiveness  of  flight  simulators  may  be  optimized  by  Integrating  them  Into  a 
flying  training  program  in  a  number  of  different  ways. 

5.  Successful  assessment  of  aircrew  performance  has  been  demonstrated  in  state  of  the  art 
simulators.  Such  performance  measures  may  be  used  to  evaluate  the  progress  of  Individual  aircrew 
trainees  and  to  evaluate  the  effectiveness  of  the  training  program  Itself. 

6.  The  flight  simulators  which  evolve  from  today's  devices  will  be  organized  around  and 
dependent  on  high-performance  digital  computer  systems  which  control  the  training  environment  and 
mediate  the  interaction  between  Instructor  personnel  and  the  trainees. 
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14.Abstract 

The  reduced  availability  and  high  cost  of  fuel  has  led  to  increased  emphasis  on  the  use  of 
simulators  for  in-flight  and  emergency  procedures  training. 

The  nature  of  pilot  skills  is  examined  and  the  report  suggests  a  strategy  for  the  use  of 
simulators  to  acquire  and  maintain  them. 

The  salient  characteristics  of  the  latest  flight  simulator  and  the  visual  display  system  capabili¬ 
ties  they  provide  for  training  contact  flying  tasks  are  described.  Research  on  transfer  of  learn¬ 
ing  from  simulator  to  aircraft  in  a  variety  of  training  tasks  is  reviewed.  The  use  of  simulators 
as  aircraft  substitutes,  and  their  integration  with  other  ground  training  media  are  composed  and 
contrasted  to  illustrate  the  cost-effectiveness  potential  of  these  two  approaches. 

The  rationale  for  and  the  characteristics  of  a  simulator-oriented  emergency  procedures  training 
programme  which  emphasises  pilot  decision  making  skills  are  presented. 

The  development  of  aircrew  performance  measurement  systems  for  use  in  evaluating,  refining 
and  documenting  simulator  training  effectiveness  is  reported. 

The  AGARDograph  ends  with  an  interpretation  of  the  impact  of  current  developments  on  the 
future  use  of  simulators;  a  list  of  conclusions  is  provided. 

This  AGARDograph  was  prepared  at  the  request  of  the  Aerospace  Medical  Panel  of  AGARD/ 
NATO. 
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